Wow, lots of people submitted test results for 1.0rc2 already! I ran my skipgood filter on it, put the digested data up at http://kegel.com/wine/failing-1.0-rc2.html and added a little analysis to http://wiki.winehq.org/MakeTestFailures To wit:
-- snip ---
Tests that fail on most versions of Windows
As of 1.0-rc2, the following tests all fail on 2000, XP, 2003, and Vista:
comctl32:listview ddraw:d3d gdi32:metafile kernel32:toolhelp mshtml:htmldoc oleaut32:olefont
and a bunch more fail on most of those versions of Windows. See http://kegel.com/wine/failing-1.0-rc2.html for the full data.
Most Common Failures on XP
As of 1.0-rc2, the following tests fail on all machines running XP:
comctl32:listview ddraw:d3d gdi32:metafile kernel32:actctx kernel32:toolhelp mshtml:htmldoc msvcrt:cpp oleaut32:olefont urlmon:url user32:input user32:listbox user32:msg
and a whole mess fail for nearly all reporters. See http://kegel.com/wine/failing-1.0-rc2.html for the full data.
Most Common Failures on Wine
As of 1.0-rc2, here are the tests that fail on some Wine systems, and how many people they fail for:
14 user32:msg 12 user32:win 9 user32:input 6 dsound:ds3d dsound:ds3d8 kernel32:path user32:monitor 5 kernel32:process msi:install msi:package riched20:editor 4 comctl32:listview ddraw:visual dinput:device gdi32:font gdi32:metafile kernel32:profile kernel32:sync kernel32:virtual snmpapi:util urlmon:protocol user32:class wininet:internet 3 comctl32:monthcal ddraw:d3d kernel32:volume msi:msi setupapi:devinst shell32:shlfolder user32:menu userenv:userenv 2 advapi32:registry advapi32:security advapi32:service comctl32:tooltips d3d8:visual d3d9:visual ddraw:dsurface dsound:dsound kernel32:version msvcrt:time ntdll:exception psapi:psapi_main qedit:mediadet 1 advpack:advpack advpack:install ...[many deleted]... wininet:http winmm:wave winspool.drv:info
Whew, that's a lot of data. I looked at all the ones that failed for five or more people:
user32:msg, user32:win, and user32:input are probably people running non-blessed window managers; Alexandre's position is that we can ignore these, since upstream gnome's WM is fixed.
dsound:ds3d is mostly "ds3d.c:891: Test failed: The primary volume changed from -27 to -960" (worrisome, random) plus "ds3d.c:896: Test failed: The primary pan changed from -5594 to -5595" (rounding error, just add slop of 1?). dsound:ds3d8 is the same.
kernel32:path is mostly "path.c:899:TMP=c:\windows\temp"..."path.c:1178: Test failed: expected buf[0] upper case letter got c", probably people whose ~/.wine is old and has a windir that starts with a lowercase drive letter?! Has that changed recently?
msi:install is mostly "install.c:4414: Test failed: File installed".
msi:package is mostly "package.c:1661: Test failed: wrong return val (0)"
riched20:editor has several failures. --- snip ---
kernel32:path is mostly "path.c:899:TMP=c:\windows\temp"..."path.c:1178: Test failed: expected buf[0] upper case letter got c", probably people whose ~/.wine is old and has a windir that starts with a lowercase drive letter?! Has that changed recently?
I don't think it can be a stale .wine; presuming those were all done with dotests, it creates a new wineprefix for each run.
Cheers,
Jeremy
On Sat, May 24, 2008 at 7:49 PM, Jeremy White jwhite@winehq.org wrote:
kernel32:path is mostly "path.c:899:TMP=c:\windows\temp"..."path.c:1178: Test failed: expected buf[0] upper case letter got c", probably people whose ~/.wine is old and has a windir that starts with a lowercase drive letter?! Has that changed recently?
I don't think it can be a stale .wine; presuming those were all done with dotests, it creates a new wineprefix for each run.
The guys with those failures aren't running dotest, darn it.
Figured out why so many people are running winetest on windows today: http://linux.slashdot.org/comments.pl?sid=563226&cid=23530662
Added James' comments to the wiki page, thanks.
On Sat, May 24, 2008 at 8:18 PM, Dan Kegel dank@kegel.com wrote:
Figured out why so many people are running winetest on windows today: http://linux.slashdot.org/comments.pl?sid=563226&cid=23530662
I wanted to reply to that post, but /. locked me out. If anyone else wants to do it, here's what I was going to say:
--- snip ---
See also <a href="http://wiki.winehq.org/ConformanceTests">http://wiki.winehq.org/ConformanceTests</a> and <a href="http://wiki.winehq.org/MakeTestFailures">http://wiki.winehq.org/MakeTestFailures</a>
For those wondering where the latest data is: in <a href="http://test.winehq.org">http://test.winehq.org</a>, click on the "Last Modified" column twice, that will bring the latest data to the top.
Thanks to everyone who submitted data so far! We have enough reports for XP now, but any other version of Windows would be handy.
Be sure to run this again when wine-1.0-rc3 comes out next week.
I wanted to reply to that post, but /. locked me out. If anyone else wants to do it, here's what I was going to say:
They shifted to new servers tonight, probably had something to do with it.
I plagiarized your post completely.
Cheers,
Jeremy
Dan Kegel wrote:
On Sat, May 24, 2008 at 8:18 PM, Dan Kegel dank@kegel.com wrote:
Figured out why so many people are running winetest on windows today: http://linux.slashdot.org/comments.pl?sid=563226&cid=23530662
I wanted to reply to that post, but /. locked me out. If anyone else wants to do it, here's what I was going to say:
--- snip ---
See also <a href="http://wiki.winehq.org/ConformanceTests">http://wiki.winehq.org/ConformanceTests</a> and <a href="http://wiki.winehq.org/MakeTestFailures">http://wiki.winehq.org/MakeTestFailures</a>
For those wondering where the latest data is: in <a href="http://test.winehq.org">http://test.winehq.org</a>, click on the "Last Modified" column twice, that will bring the latest data to the top.
There is no need to do the last modified thingy. Alexandre changed that a few week ago, it should be the default now to list the newest test results first.
On Sunday 25 May 2008 05:18:18 Dan Kegel wrote:
Figured out why so many people are running winetest on windows today: http://linux.slashdot.org/comments.pl?sid=563226&cid=23530662
Yes, this was a surprise.
IIRC, these tests are considered slightly hazardous. In the past we recommend that only developers (and others who understand the slight risk) run winetest.exe; although obviously the greater coverage is useful.
Before people run the tests, I feel we should give them the opportunity[1] to find out: a. what the tests do and why it's useful, b. who should help (e.g., answering the question: how about my Win. install inside VMWare?). c. what might go wrong (e.g., stale Reg. entries, computer freezing), how likely these problems are to occur and how to fix them if they do happen,
I'd recommend we add a clearly worded link at the top of each results page, that directs the person to a wiki page with above information. For example, this could be the existing ConformanceTests wiki; although, with the additional information, that wiki might be a bit too busy.
Also, it would be nice if we provided some way that end-users could verify the authenticity of winetest.exe. I PGP/GPG sign the binaries, but currently those don't propagate to the test-result page, so people probably won't see 'em.
Also, detached PGP signatures aren't very handy. I guess running codesign.exe (or equiv[2]) would be better, but I've not used it before and we'd have to think about the X509 setup.
Cheers,
Paul.
[1] -- also surprising was that one post to Slashdot and a legit-looking URI and many people downloaded and ran winetest.exe. People's laissez-faire attitude towards downloading and running executable content off the Internet helps keep the anti-virus business going, I guess :-/
[2] -- A quick search found this: http://sourceforge.net/projects/osslsigncode/ I don't know how well it works.
On Sat, May 24, 2008 at 9:41 PM, Dan Kegel dank@kegel.com wrote:
Wow, lots of people submitted test results for 1.0rc2 already! I ran my skipgood filter on it, put the digested data up at http://kegel.com/wine/failing-1.0-rc2.html and added a little analysis to http://wiki.winehq.org/MakeTestFailures To wit:
-- snip ---
Tests that fail on most versions of Windows
As of 1.0-rc2, the following tests all fail on 2000, XP, 2003, and Vista:
comctl32:listview ddraw:d3d gdi32:metafile kernel32:toolhelp mshtml:htmldoc oleaut32:olefont
I'm gonna break this list up to make it easier to read:
comctl32:listview * fixed by this patch, http://winehq.org/pipermail/wine-patches/2008-May/055110.html
ddraw:d3d * result of vmware driver bugs, most of which have been ferreted out, but I'm leaving these up to Stefan D. for now
gdi32:metafile * Dmitry's previous patch brought the number of test failures down from ~10 to 1, so I'm hoping he can get this last one next
kernel32:toolhelp * see the discussion at, http://winehq.org/pipermail/wine-devel/2008-April/065040.html
mshtml:htmldoc * I sent in a patch for a few failures in win2k3, have to ask Jacek C. about the other two: http://winehq.org/pipermail/wine-patches/2008-May/055087.html
oleaut32:olefont * I've been in private discussion with Walt Ogburn about these tests. He swears they passed when he wrote them, and he doesn't know why they don't pass now. If he or someone else can't find a fix, I think the test should be removed because we're conforming to a behavior that doesn't match any windows version.
and a bunch more fail on most of those versions of Windows. See http://kegel.com/wine/failing-1.0-rc2.html for the full data.
Most Common Failures on XP
As of 1.0-rc2, the following tests fail on all machines running XP:
comctl32:listview ddraw:d3d gdi32:metafile kernel32:actctx kernel32:toolhelp mshtml:htmldoc msvcrt:cpp oleaut32:olefont urlmon:url user32:input user32:listbox user32:msg
kernel32:actctx * fixed by a patch I'm sending in Sunday night
user32:input * had a private discussion with Vitaliy. He said he wrote the tests to pass in win2k, doesn't care that they don't pass for other platforms
oleaut32:olefont
- I've been in private discussion with Walt Ogburn about these tests.
He swears they passed when he wrote them, and he doesn't know why they don't pass now. If he or someone else can't find a fix, I think the test should be removed because we're conforming to a behavior that doesn't match any windows version.
You can test these ones easily on Linux with a native oleaut32.dll btw using
WINEDLLOVERRIDES=oleaut32=n wine oleaut32_test.exe.so olefont
Ciao, Marcus
-
Dan Kegel -
James Hawkins -
Jeremy White -
Marcus Meissner -
Paul Millar -
Paul Vriens