On 10/25/07, Francois Gouget fgouget@free.fr wrote:
Here are some things I noticed while using this site. Let me know if I it would help to make bug reports for these:
- Some result entries are red with a dash in them and a blue border.
See the Windows 98 results for http://test.winehq.org/data/200710241000/ I assume these means that the test did not load. However we should distinguish two cases there:
- if it did not run because the tested dll did not exist at all, then
it's not a test failure and thus the background should be green. A typical case would be the crypt32 tests on Windows 98.
- if the dll was there but the test still did not run, typically
because the dll is missing an export, then that's a bug in the test: it should dynamically load that function so the other checks it performs can be run. A typical case is the gdi32 tests on Windows 98. Ideally we'd even have a log showing the missing API but that's probably too tricky to do on Windows.
- Downloading the log file for a given test run gives me a file that's
called 'report'. It would be nice if it was called something like 'vmware-win98-report' instead so that saving a couple of them in a directory leads to fewer collisions.
- It would be nice if the /data page looked more like a calendar with
the most current date easily accessible, and a less like the directory listing it currently is. Then a / page would be nice too.
I'm pretty unlikely to fix these, but at least here is a starting if someone is inclined to do so: you will find the source for the test.winehq.org website in the tools.git repository: http://source.winehq.org/git/tools.git/?a=tree;f=winetest
To get hold of the sources, see the instructions there: http://www.winehq.org/site/git#modules
Looking at the test data, all of the msi:install tests timeout. I just ran the install tests in XP running under vmware on a 3ghz machine. The tests took 9m41s. That completely blows away the 2min timeout. There's nothing wrong with the tests, they just take a long time. I don't think we should extend the timeout, because it's very subjective and more tests will be added, meaning we'll have to change the timeout eventually. I do think we should have a flag or variable that allows the timeout to be ignored for certain tests. Any opinions?