Documenting WineTest best practices

[Francois Gouget]

I have done a major refresh of the Wiki's conformance tests page.
http://wiki.winehq.org/ConformanceTests

One aspect I would like to document there is the best practices mostly 
for people running WineTest unattended on their computer. For instance I 
recently learned that one should really make sure that ntlm_auth works 
otherwise there will be failures.

There are other things for which I don't have the answer however. In 
particular:

* On Windows XP the firewall will ask whether to block network access 
   to some tests. How should this be dealt with. Since WineTest is being 
   run unattended clicking 'ok' does not work.

* Similarly on Vista and Windows 7 UAC comes into play. I believe it 
   mostly causes some tests to be skipped. So again, are there 
   recommandations around this?

* Are we interested in tests being run as an administrator or a regular 
   user? Both?

I also came across the MakeTestFailures page and it seems pretty out of 
date. Is maintaining a list of failing tests (still) really useful?

http://wiki.winehq.org/MakeTestFailures

[Francois Gouget]

On Wed, 24 Aug 2011, Francois Gouget wrote:
[...]
> -
>
> Similarly on Vista and Windows 7 UAC comes into play. I believe it 
> mostly causes some tests to be skipped. So again, are there 
> recommandations around this?
> -
>
> Are we interested in tests being run as an administrator or a regular 
> user? Both?

On my Windows 7 VM WineTest is not running with elevated privileges. 
This is what causes the advpack:install test to time out:

http://test.winehq.org/data/6a6aab27633405aeb5e464943110960f0099dffe/index_W...

The reason is that the calls listed below cause Windows 7 to bring up a 
dialog telling me I need administrator privileges:

You do not have administrator privileges on this machine. This 
   installation cannot be completed correctly unless it is run by an 
   administrator.

Furthermore I get this issue even if I set UAC to 'Never notify'.

I think the tests should work in both elevated and non-elevated 
privilege accounts. So this probably means detecting the non-elevated 
case and skipping some tests. Or finding a way to tell Windows not to 
bring up that dialog and just fail.

hr = pRunSetupCommand(NULL, path, "DefaultInstall", dir, "Title", NULL, 
                      RSC_FLAG_INF | RSC_FLAG_QUIET, NULL);

hr = pRunSetupCommand(NULL, path, "DefaultInstall", "", "Title", NULL, 
                      RSC_FLAG_INF | RSC_FLAG_QUIET, NULL);

hr = pRunSetupCommand(NULL, "one\test.inf", "DefaultInstall", dir, 
                      "Title", NULL, RSC_FLAG_INF | RSC_FLAG_QUIET, NULL);

hr = pRunSetupCommand(NULL, "one\test.inf", "DefaultInstall", dir, 
                      "Title", NULL, RSC_FLAG_INF | RSC_FLAG_QUIET, NULL);

hr = pRunSetupCommand(NULL, "one\test.inf", "DefaultInstall", "", 
                      "Title", NULL, RSC_FLAG_INF | RSC_FLAG_QUIET, NULL);

hr = pLaunchINFSection(NULL, NULL, cmdline, 0);

hr = pLaunchINFSection(NULL, NULL, file, 0);

hr = pLaunchINFSectionEx(NULL, NULL, cmdline, 0);