Some general questions about test scripts

[Paul Vriens]

Hi,

I'm currently working on the shlwapi tests and I already put something
out on wine-devel for that.

I tried a test on several operating systems with different dll's
(out-of-the-box installation, servicepacks, Windows Update) so the end
result is I have several different outputs of the test.

I now more or less have the following (I'm exaggerating):

ok( a < b | a == b | a > b, "Test failed\n");

So the question is how do we deal with the above? If the number of
possible 'correct' output is huge, what's the benefit of testing?

The second question is what should we implement in Wine?

So much for the general questions. Now for the specific one:

The test I'm referring to is SHQueryValueExA that I modified a bit for
testing purposes:

for (counter = 0; counter <= sExpLen2 ; counter++)
{
  strcpy(buf, sEmptyBuffer);
  dwSize = counter;
  trace("Now trying with dwSize=(%lu)\n", counter);
  dwType = -1;
  dwRet = SHQueryValueExA( hKey, "Test3", NULL, &dwType, buf, &dwSize);
  ok( ERROR_MORE_DATA == dwRet, "Expected ERROR_MORE_DATA, got (%lu)\n",
dwRet);
  trace("comparing (%s) with (%s)\n", buf, sEmptyBuffer);
  ok( 0 == strcmp(sEmptyBuffer, buf) , "Comparing (%s) with (%s)
failed\n", buf, sEmptyBuffer);
  ok( dwSize >= nUsedBuffer2, "Buffer size (%lu) should be >= (%lu)\n",
dwSize, nUsedBuffer2);
  ok( REG_SZ == dwType , "Expected REG_SZ, got (%lu)\n", dwType);
}

I tested this on 3 different versions of win98, winxp-prof, winxp-home,
winnt-server, w2k-prof and w2k-server and Wine of course.

I can see that winxp/w2k and winnt act the same. Win98 and Wine do it
differently (Wine doesn't do it as win98 does btw).

Any thoughts?

Paul Vriens.