winetest on wine and todo_wine

[Paul Vriens]

Hi,

in shlwapi/tests/shreg.c I've put two todo_wine statements for obvious
reasons (they fail, so do not test).

If I however look at
http://test.winehq.org/data/200502101000/wine_98/shlwapi:shreg.txt

These tests are executed! Any ideas what could be wrong?

If I execute the tests on my Wine box the tests are skipped.

Cheers,

Paul Vriens.

[Mike Hearn]

On Thu, 10 Feb 2005 20:13:21 +0100, Paul Vriens wrote:
> These tests are executed! Any ideas what could be wrong?
>
> If I execute the tests on my Wine box the tests are skipped.

You misunderstood what the meaning of todo_wine is - it doesn't mean
"these tests aren't quite right yet", it means "wine is buggy in this
place". So yes the tests will run on Windows because it's assumed that
every test case proves some fact about the way Windows works. Todo_wine
just stops the test suite complaining if Wine is known to be buggy.

thanks -mike

[Paul Vriens]

On Thu, 2005-02-10 at 20:36, Mike Hearn wrote:
> On Thu, 10 Feb 2005 20:13:21 +0100, Paul Vriens wrote:
> > These tests are executed! Any ideas what could be wrong?
> >
> > If I execute the tests on my Wine box the tests are skipped.
>
> You misunderstood what the meaning of todo_wine is - it doesn't mean
> "these tests aren't quite right yet", it means "wine is buggy in this
> place". So yes the tests will run on Windows because it's assumed that
> every test case proves some fact about the way Windows works. Todo_wine
> just stops the test suite complaining if Wine is known to be buggy.
>
> thanks -mike

So todo_wine skips the tests on Wine, yes. That's my point.

The link is was referring to was a test on Wine.

If I run the shlwapi/shreg tests on Wine I get:

[paul@penguin tests]$ ../../../tools/runtest -v -P wine -M shlwapi.dll
-T ../../.. -p shlwapi_test.exe.so shreg.c && touch shreg.ok
shreg.c:97:sExplen1 = (12)
shreg.c:99:sExplen2 = (43)
shreg.c: 45 tests executed, 2 marked as todo, 0 failures.

which looks fine to me.

Why are these not skipped on:

http://test.winehq.org/data/200502101000/wine_98/shlwapi:shreg.txt

Cheers,

Paul.

[Michael Stefaniuc]

Paul Vriens wrote:
> On Thu, 2005-02-10 at 20:36, Mike Hearn wrote:
> > On Thu, 10 Feb 2005 20:13:21 +0100, Paul Vriens wrote:
> > > These tests are executed! Any ideas what could be wrong?
> > >
> > > If I execute the tests on my Wine box the tests are skipped.
> >
> > You misunderstood what the meaning of todo_wine is - it doesn't mean
> > "these tests aren't quite right yet", it means "wine is buggy in this
> > place". So yes the tests will run on Windows because it's assumed that
> > every test case proves some fact about the way Windows works. Todo_wine
> > just stops the test suite complaining if Wine is known to be buggy.
> >
> > thanks -mike
>
> So todo_wine skips the tests on Wine, yes. That's my point.

No, the todo_wine tests dosn't skip the tests on Wine. It just ignores 
them if they fail. If they succeed, it will tell you that you can remove 
the todo_wine.
> The link is was referring to was a test on Wine.
>
> If I run the shlwapi/shreg tests on Wine I get:
>
> [paul@penguin tests]$ ../../../tools/runtest -v -P wine -M shlwapi.dll
> -T ../../.. -p shlwapi_test.exe.so shreg.c && touch shreg.ok
> shreg.c:97:sExplen1 = (12)
> shreg.c:99:sExplen2 = (43)
> shreg.c: 45 tests executed, 2 marked as todo, 0 failures.
>
> which looks fine to me.
>
> Why are these not skipped on:
>
> http://test.winehq.org/data/200502101000/wine_98/shlwapi:shreg.txt

Because that's a Windows box and not Wine. todo_wine isn't for wrong 
tests which need to be fixed. If you need that functionality you have to 
comment the tests out.

bye
    michael

[Mike Hearn]

On Thu, 10 Feb 2005 21:00:44 +0100, Paul Vriens wrote:
> Why are these not skipped on:
>
> http://test.winehq.org/data/200502101000/wine_98/shlwapi:shreg.txt

Well, the test.winehq.org site isn't meant to receive Wines test results,
it's for collecting Windows test results. Presumably the way winrash runs
the tests means that todo_wine blocks aren't skipped, but I'm not sure why
we have Wine results on this site anyway ...

[Jakob Eriksson]

Mike Hearn wrote:
> On Thu, 10 Feb 2005 21:00:44 +0100, Paul Vriens wrote:
> > Why are these not skipped on:
> >
> > http://test.winehq.org/data/200502101000/wine_98/shlwapi:shreg.txt
>
> Well, the test.winehq.org site isn't meant to receive Wines test results,
> it's for collecting Windows test results. Presumably the way winrash runs
> the tests means that todo_wine blocks aren't skipped, but I'm not sure why
> we have Wine results on this site anyway ...

Only because some pesky person will inevitably run winetest.exe
on Wine instead of Windows.  Just to see what happens.

I know I did. :-)

(Plus it's kind of nice to have a Wine list just right to the Windows 
results.)

regards,
Jakob

[Mike Hearn]

Jakob Eriksson wrote:
> Only because some pesky person will inevitably run winetest.exe
> on Wine instead of Windows.  Just to see what happens.
>
> I know I did. :-)

Heh, well, in my mind that's another reason to disable it for the Linux 
build. Given that assembling it flattens my computer (with "only" 256mb 
of ram), and that it's being built for the crosstests, I don't think 
there's any disadvantage to doing so.

[Ferenc Wagner]

Paul Vriens  writes:
> in shlwapi/tests/shreg.c I've put two todo_wine statements
> for obvious reasons (they fail, so do not test).
>
> If I however look at
> http://test.winehq.org/data/200502101000/wine_98/shlwapi:shreg.txt
>
> These tests are executed! Any ideas what could be wrong?

Yep.  Winetest resets all environmental variables
influencing the tests unless the -e option is given.  It is
to provide a consistent environment for the tests, like
tuning verbosity and going noninteractive.  I'm not sure
what happens if WINETEST_PLATFORM or the others are simply
unset; you are welcome to persuade me or change the code.