Hi,
I execute Wine tests on clean Windows XP SP2 and Windows 2003 SP1 EE.
I tried to use both pre-compiled binary from http://www.astro.gla.ac.uk/users/paulm/WRT/CrossBuilt/winetest-latest.exe and Wine sources built by myself with Visual Studio .NET 2003 (20050830 version).
What I do is just launching the test several times on the same machine and comparing log files received.
Please, see the sample diff of 2 such log files.
Why the number of tests executed and number of failures are so different for the identical test groups?
It seems to be some synchronization problem.
How should I fix this to obtain stable results?
Thank you in advance.
------------------------------------------------------------ @@ -50,7 +50,7 @@ module: 34 tests executed, 0 marked as todo, 0 failures. mailslot: 75 tests executed, 0 marked as todo, 0 failures. path: 1741 tests executed, 0 marked as todo, 0 failures. -pipe: 419 tests executed, 0 marked as todo, 0 failures. +pipe: 418 tests executed, 0 marked as todo, 0 failures. tests/process.c: 1 tests executed, 0 marked as todo, 0 failures. tests/process.c: 1 tests executed, 0 marked as todo, 0 failures. tests/process.c: 1 tests executed, 0 marked as todo, 0 failures. @@ -78,7 +78,7 @@ util: 139 tests executed, 0 marked as todo, 136 failures. mlang: 11211 tests executed, 0 marked as todo, 20 failures. msacm: 385 tests executed, 0 marked as todo, 0 failures. -profile: 127 tests executed, 0 marked as todo, 1 failure. +profile: 127 tests executed, 0 marked as todo, 0 failures. htmldoc: 688 tests executed, 0 marked as todo, 0 failures. protocol: 255 tests executed, 0 marked as todo, 0 failures. db: 73 tests executed, 0 marked as todo, 0 failures. @@ -104,7 +104,7 @@ env: 611 tests executed, 0 marked as todo, 0 failures. error: 813 tests executed, 0 marked as todo, 0 failures. generated: 1981 tests executed, 0 marked as todo, 0 failures. -info: 765 tests executed, 0 marked as todo, 0 failures. +info: 725 tests executed, 0 marked as todo, 0 failures. large_int: 700 tests executed, 0 marked as todo, 0 failures. path: 230 tests executed, 0 marked as todo, 5 failures. reg: 9 tests executed, 0 marked as todo, 0 failures. @@ -133,7 +133,7 @@ main: 15 tests executed, 0 marked as todo, 1 failure. generated: 1141 tests executed, 0 marked as todo, 0 failures. shelllink: 116 tests executed, 0 marked as todo, 3 failures. -tests/shellpath.c: 10 tests executed, 0 marked as todo, 0 failures. +tests/shellpath.c: 10 tests executed, 0 marked as todo, 4 failures. tests/shellpath.c: 4 tests executed, 0 marked as todo, 0 failures. shellpath: 447 tests executed, 0 marked as todo, 3 failures. shlexec: 20 tests executed, 0 marked as todo, 0 failures. @@ -174,4 +174,4 @@ timer: 44 tests executed, 0 marked as todo, 0 failures. wave: 289 tests executed, 0 marked as todo, 15 failures. protocol: 26 tests executed, 0 marked as todo, 0 failures. -sock: 8971 tests executed, 0 marked as todo, 5 failures. +sock: 8951 tests executed, 0 marked as todo, 5 failures. ------------------------------------------------------------
-- Sincerely yours, katy