http://bugs.winehq.org/show_bug.cgi?id=12074
Summary: The conformance tests fail on Windows Product: Wine Version: unspecified Platform: Other URL: http://test.winehq.org/data/ OS/Version: other Status: NEW Severity: enhancement Priority: P2 Component: testcases AssignedTo: wine-bugs@winehq.org ReportedBy: fgouget@codeweavers.com
The conformance tests, that is what constitutes 'make test', are supposed to test and document the Windows behavior so that we can then verify that Wine behaves like Windows. This has two important consequences: * it means that no conformance test should ever fail on Windows. * it means that any conformance test that fails on Windows is useless for making sure that Wine behaves like Windows.
Unfortunately we currently have too many conformance tests that fail on Windows. As of today (2008/03/16), approximately 37% of the conformance tests fail on Windows XP, 2003 or Vista. If you add other Windows versions, then this percentage rises well above 50%.
See the URL below for up-to-date results: http://test.winehq.org/data/
And for information on how to run the conformance tests:
http://wiki.winehq.org/ConformanceTests http://www.winehq.org/docs/en/winedev-guide.html#CROSS-COMPILING-TESTS http://www.winehq.org/docs/en/winedev-guide.html#TESTING-WINDOWS http://www.astro.gla.ac.uk/users/paulm/WRT/CrossBuilt/winetest-latest.exe
http://bugs.winehq.org/show_bug.cgi?id=12074
François Gouget fgouget@codeweavers.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|--- |1.0.0
http://bugs.winehq.org/show_bug.cgi?id=12074
Austin English austinenglish@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|1.0.0 |1.2.0
--- Comment #1 from Austin English austinenglish@gmail.com 2008-04-20 14:36:55 --- There's no way we'll have this fixed by 1.0.
http://bugs.winehq.org/show_bug.cgi?id=12074
James Hawkins truiken@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|1.2.0 |1.0.0
--- Comment #2 from James Hawkins truiken@gmail.com 2008-04-20 14:45:56 --- I disagree. You set up a blocker list for a release so that developers will knuckle down in order to get the release out on time. Getting soft defeats the purpose. If 5 developers stepped up and fixed 10 win2k3 tests each, the win2k3 tests failures would be 0. We have 54 days left to fix this, so give it a chance. If we get to the release date and it's not fixed, then we'll defer it.
http://bugs.winehq.org/show_bug.cgi?id=12074
nathan.n saturn_systems@yahoo.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |saturn_systems@yahoo.com
http://bugs.winehq.org/show_bug.cgi?id=12074
shakaran shakaran@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |shakaran@gmail.com
http://bugs.winehq.org/show_bug.cgi?id=12074
Alexandre Julliard julliard@winehq.org changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |juan_lang@yahoo.com
--- Comment #3 from Alexandre Julliard julliard@winehq.org 2008-06-05 03:59:03 --- *** Bug 13700 has been marked as a duplicate of this bug. ***
http://bugs.winehq.org/show_bug.cgi?id=12074
Dan Kegel dank@kegel.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |dank@kegel.com Target Milestone|1.0.0 |1.2.0
--- Comment #4 from Dan Kegel dank@kegel.com 2008-06-11 16:08:16 --- We made some progress on this, thanks especially to James. I think subtantial fixes to our conformance test suite will have to wait until after 1.0, though, so I'm deferring this.
http://bugs.winehq.org/show_bug.cgi?id=12074
Austin English austinenglish@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |austinenglish@gmail.com
--- Comment #5 from Austin English austinenglish@gmail.com 2008-06-11 16:46:38 --- Many thanks to James for all those tests fixes. Looking at test.winehq.org is now _MUCH_ easier on the eyes.
I've noticed a bit on wine-devel a bit of discussion on version checking, and I'm debating filing a bug for it, but haven't yet. It seems that since so much of windows behavior changes by version, that instead of doing:
--pseudo code-- ok(!result && (GetLastError() == ERROR_A|| GetLastError() == ERROR_B)
That we should be doing
if (win_version == WIN_98) then ((GetLastError() == ERROR_A)) elseif (win_version == WIN_2K) then ((GetLastError() == ERROR_B))
etc...
While not as clean of an implementation, neither is the reset of win32 API. It would allow for a truer test of windows functions, as we'd be able to ensure that we're conforming to the _right_ windows behavior rather than _some form_ of windows behavior.
http://bugs.winehq.org/show_bug.cgi?id=12074
--- Comment #6 from James Hawkins truiken@gmail.com 2008-06-11 17:40:15 ---
While not as clean of an implementation, neither is the reset of win32 API. It would allow for a truer test of windows functions, as we'd be able to ensure that we're conforming to the _right_ windows behavior rather than _some form_ of windows behavior.
Our implementation is the exact opposite, except in the very rare case that an app also checks the windows version. That's why we don't add version checks (except when we know an app does the same thing).
http://bugs.winehq.org/show_bug.cgi?id=12074
Austin English austinenglish@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Keywords| |download
--- Comment #7 from Austin English austinenglish@gmail.com 2008-12-11 10:37:47 --- Most definitely still present.
http://bugs.winehq.org/show_bug.cgi?id=12074
--- Comment #8 from Dan Kegel dank@kegel.com 2008-12-11 10:52:36 --- Can we get the graphics card reported by winetest? I wonder if our failure rate is higher on certain graphics cards even on windows.
http://bugs.winehq.org/show_bug.cgi?id=12074
--- Comment #9 from Jeff Zaroyko jeffz@jeffz.name 2008-12-11 16:13:33 --- (In reply to comment #8)
Can we get the graphics card reported by winetest? I wonder if our failure rate is higher on certain graphics cards even on windows.
both the opengl test and the d3d9/visual test should print some info about the video card:
dlls/d3d9/tests/visual.c visual.c:183: Driver string: "nvd3dum.dll" visual.c:184: Description string: "NVIDIA GeForce 8800 GTS"
dlls/opengl32/tests/opengl.c opengl.c:581: OpenGL renderer: GeForce 8800 GTS/PCI/SSE2
http://bugs.winehq.org/show_bug.cgi?id=12074
François Gouget fgouget@codeweavers.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution| |FIXED
--- Comment #10 from François Gouget fgouget@codeweavers.com 2009-11-08 03:40:00 --- Making sure the conformance tests work fine on Wine is still very important but it's now an ongoing project which gets the attention it deserves. So I think it can be closed.
As a reminder for developers; after committing a patch to a conformance test, it's a good idea to go look at the results on the different Windows versions on the winetest results page: http://test.winehq.org/
http://bugs.winehq.org/show_bug.cgi?id=12074
Alexandre Julliard julliard@winehq.org changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |CLOSED
--- Comment #11 from Alexandre Julliard julliard@winehq.org 2009-11-13 12:42:23 --- Closing bugs fixed in 1.1.33.
-
wine-bugs＠winehq.org