Hi Hans,
Which conformance tests fail currently? I don't see any problems with the kernel DLL tests right at the moment, which is somewhat disconcerting.
I do see a problem with the listbox tests (in the user dll tests). There's also various bugs in pipe tests (under kernel) that mysteriously vanished after closing my eyes and counting to ten: http://www.astro.gla.ac.uk/users/paulm/WRT/wrt.php
Anyone have any idea what might have happened?
Cheers,
Paul.
On Mon, 17 Nov 2003, Hans Leidekker wrote:
This makes Wine pass the kernel directory conformance test without failures.
---- Paul Millar
On Monday 17 November 2003 16:34, Paul Millar wrote:
Which conformance tests fail currently? I don't see any problems with the kernel DLL tests right at the moment, which is somewhat disconcerting.
These are the errors I got before my patch:
directory.c:199: Test failed: CreateDirectoryA with ? wildcard name should fail, ret= True error=183 directory.c:206: Test failed: CreateDirectoryA with * wildcard name should fail, ret= True error=183 directory.c:262: Test failed: CreateDirectoryW with ? wildcard name should fail with error 183, ret= True error=183 directory.c:269: Test failed: CreateDirectoryW with * wildcard name should fail with error 183, ret= True error=183 directory.c:292: Test failed: RemoveDirectoryA with ? wildcard name should fail with error 183, ret=False error=2 directory.c:298: Test failed: RemoveDirectoryA with * wildcard name should fail with error 183, ret=False error=2 directory.c:322: Test failed: RemoveDirectoryW with wildcard should fail with error 183, ret=False error=2 directory.c:328: Test failed: RemoveDirectoryW with * wildcard name should fail with error 183, ret=False error=2
But there are more kernel tests that fail for me. It depends on a lot of factors which and how many I see. I have contrasted the following combinations of compiler platform and target platform: gcc compiled test on Wine1 and Wine2, MingW cross-compiled tests on Wine1 and Wine2, and MingW cross-compiled test on Win2k. Wine1 is cvs Wine with a wineinstall:ed Windows installation, Wine2 is cvs Wine plus a CXOffice Windows tree. Unfortunately there can be a lot of reasons tests fail (some might not even run or crash):
1. The test is simply incorrect (e.g. it fails on Windows). My other patch to the kernel path conformance test illustrates this.
2. The compiler and link libraries may cause differring results. For example, I get this error with a MinGW compiled kernel test:
generated.c:542: Test failed: TYPE_ALIGNMENT(*(LPWIN32_STREAM_ID)0) == 8 (expected 4)
This is because gcc doesn't align doubles (the LPWIN32_STREAM_ID structure contains a LARGE_INTEGER, which is typedef:ed long long in the Wine headers. MinGW defaults to -malign-doubles, hence the difference. I have to check what MSVC does here.
I also had problems with the comctl tests that would link with MinGW but not run. This appeared to be due to faulty MinGW import libs.
3. The test may crash. For example, the shlwapi string test crashed on CXOffice and Win2k. Older versions of MS shlwapi (I tested 5.00.2920.0000 and 5.50.4134.600) had a bug in StrChrA which causes the crash. Resolution: update shlwapi. I found that in a newer version I have (6.0.2800.1226) this bug doesn't exist anymore
4. Hangs. Some tests like the kernel pipe and kernel process tests appear to have races. They may or may not hang depending on system load and other factors.
5. Tests may fail because Wine doesn't implement some functionality yet or implements it incorrectly. This is the normal case ;-) Sometimes inserting 'todo_wine' before a test is appropriate when it's certain that it takes a long time before Wine implements some missing functionality.
-Hans
On Monday 17 November 2003 16:34, Paul Millar wrote:
Which conformance tests fail currently? I don't see any problems with the kernel DLL tests right at the moment, which is somewhat disconcerting.
These are the errors I got before my patch:
directory.c:199: Test failed: CreateDirectoryA with ? wildcard name should fail, ret= True error=183 directory.c:206: Test failed: CreateDirectoryA with * wildcard name should fail, ret= True error=183 directory.c:262: Test failed: CreateDirectoryW with ? wildcard name should fail with error 183, ret= True error=183 directory.c:269: Test failed: CreateDirectoryW with * wildcard name should fail with error 183, ret= True error=183 directory.c:292: Test failed: RemoveDirectoryA with ? wildcard name should fail with error 183, ret=False error=2 directory.c:298: Test failed: RemoveDirectoryA with * wildcard name should fail with error 183, ret=False error=2 directory.c:322: Test failed: RemoveDirectoryW with wildcard should fail with error 183, ret=False error=2 directory.c:328: Test failed: RemoveDirectoryW with * wildcard name should fail with error 183, ret=False error=2
But there are more kernel tests that fail for me. It depends on a lot of factors which and how many I see. I have contrasted the following combinations of compiler platform and target platform: gcc compiled test on Wine1 and Wine2, MingW cross-compiled tests on Wine1 and Wine2, and MingW cross-compiled test on Win2k. Wine1 is cvs Wine with a wineinstall:ed Windows installation, Wine2 is cvs Wine plus a CXOffice Windows tree. Unfortunately there can be a lot of reasons tests fail (some might not even run or crash):
1. The test is simply incorrect (e.g. it fails on Windows). My other patch to the kernel path conformance test illustrates this.
2. The compiler and link libraries may cause differring results. For example, I get this error with a MinGW compiled kernel test:
generated.c:542: Test failed: TYPE_ALIGNMENT(*(LPWIN32_STREAM_ID)0) == 8 (expected 4)
This is because gcc doesn't align doubles (the LPWIN32_STREAM_ID structure contains a LARGE_INTEGER, which is typedef:ed long long in the Wine headers. MinGW defaults to -malign-doubles, hence the difference. I have to check what MSVC does here.
I also had problems with the comctl tests that would link with MinGW but not run. This appeared to be due to faulty MinGW import libs.
3. The test may crash. For example, the shlwapi string test crashed on CXOffice and Win2k. Older versions of MS shlwapi (I tested 5.00.2920.0000 and 5.50.4134.600) had a bug in StrChrA which causes the crash. Resolution: update shlwapi. I found that in a newer version I have (6.0.2800.1226) this bug doesn't exist anymore
4. Hangs. Some tests like the kernel pipe and kernel process tests appear to have races. They may or may not hang depending on system load and other factors.
5. Tests may fail because Wine doesn't implement some functionality yet or implements it incorrectly. This is the normal case ;-) Sometimes inserting 'todo_wine' before a test is appropriate when it's certain that it takes a long time before Wine implements some missing functionality.
-Hans
On Monday 17 November 2003 18:47, Hans Leidekker wrote:
Which conformance tests fail currently? I don't see any problems with the kernel DLL tests right at the moment, which is somewhat disconcerting.
[sorry for those repeated mails, I received an error message from my mailclient and concluded that the mail hadn't been sent]
I have looked into this a bit more and I think your confusion stems from the fact that you run 'make test', which doesn't run tests marked 'todo_wine'. When you run tests by hand e.g:
wine dlls/kernel/tests/kernel32_crosstest.exe directory
those tests will run (and fail).
Which reminds me that we should remove 'todo_wine' from this test. Patch attached.
-Hans
Changelog: - Remove 'todo_wine'
On Tue, 18 Nov 2003, Hans Leidekker wrote:
On Monday 17 November 2003 18:47, Hans Leidekker wrote:
Which conformance tests fail currently? I don't see any problems with the kernel DLL tests right at the moment, which is somewhat disconcerting.
[sorry for those repeated mails, I received an error message from my mailclient and concluded that the mail hadn't been sent]
I have looked into this a bit more and I think your confusion stems from the fact that you run 'make test', which doesn't run tests marked 'todo_wine'.
This is incorrect. 'make test' runs all tests. It expects normal tests to succeed and todo_wine tests to fail. It will complain if a normal test fails or if a todo_wine succeeds.
This may still explain the confusion.
On Tue, 18 Nov 2003, Francois Gouget wrote:
On Tue, 18 Nov 2003, Hans Leidekker wrote:
I have looked into this a bit more and I think your confusion stems from the fact that you run 'make test', which doesn't run tests marked 'todo_wine'.
This is incorrect. 'make test' runs all tests. It expects normal tests to succeed and todo_wine tests to fail. It will complain if a normal test fails or if a todo_wine succeeds.
Yep, that is my understanding. Although, the complaint when a todo_wine succeeds is only a warning: the test is still marked as having passed. For example:
[...] listbox.c:129: Test succeeded listbox.c:134: Test succeeded inside todo block: style 0x4000, step step, field caret: expected 2, got 2 listbox.c:134: Test succeeded [...]
This may still explain the confusion.
Pretty much ...
[...]
wine dlls/kernel/tests/kernel32_crosstest.exe directory
This doesn't work for me... (isn't _crosstest from compiling with a cross-compiler?). The nearest I get that does something is: ./wine dlls/kernel/tests/kernel32_test.exe.so directory which *does* produce 8 failures as you described.
Clearing things up (at least, in my own mind), this only happens because you don't set the platform to be "wine". For example: WINETEST_PLATFORM=wine ./wine dlls/kernel/tests/kernel32_test.exe.so directory
Not setting WINETEST_PLATFORM to "wine" has the same effect as moving those tests outside of the wine_todo block, hence the errors.
Cheers,
Paul.
---- Paul Millar
-
Francois Gouget -
Hans Leidekker -
Paul Millar