Hi,
I apologize for the numerous occurrances of red color on test.winehq due to my new winmm MCI tests. That was not expected.
However, turning something green is not a goal in itself. For some time now I wanted to query wine-devel about this topic, independently of my patches, so here it goes.
An analysis reveals that my tests implement a detector of vmware (virtualbox etc.) default installs, i.e. MS-Windows systems that have not been configured with sound.
No real user's machines would be like that (except for a w95 box I tried with a broken soundcard).
The revealing line is
Tests skipped: Please install audio driver. Tests will fail (18 errors)
This goes hand in hand with "5 tests, 0 errors, 1 skip" in winmm:wave. Exactly those systems skipping the wave tests produce 18-19 MCI errors.
Failure to flag an error in such cases has known drawbacks: - For several month, Austin did not notice that his BSD boxes had lost sound. wave.ok simply reports "5 tests ok, 1 skipped" in joyfull green, whereas it executed many tests half a year ago.
- All those vmware boxes configured without sound and thus skipping all but 5 wave.ok tests have for monthes *not* provided valuable feedback to Wine. So they were actually *not* executing tests, when they could have. It's only when Paul tested one of my tests that he realized that sound was not configured on one of his vmware boxes, promptly configured the sound device, and the tests passed.
The picture on test.winehq.org is a disaster: None of the w95, w98, nt4, 2k, 2k3, Vista, 2k8, w7 machines have performed winmm:wave tests! They all have no sound configured. Mostly useless.
Therefore, I believe it's sometimes better to flag an error than to calmly skip something.
Last year already, submitting the ICCVID patches, I was thinking: "iccvid.dll may not part of every installation, but we want feedback so we should compare behaviour with MS-Windows systems equipped with that library and ask people to install it if not present". Likewise, Wine should compare with systems similarly equipped: d3d8, d3d9, dplay, dmusic etc.
Perhaps an easy change on test.winehq would be to augment the blue color indicator of skipped tests to the full rectangle surface instead of the current green?
Of course, I'll change my tests to flag a single error on systems without sound. -- Or no error at all, if that's wine-devel consensus.
What do you think? Jörg Höhle. There's only few real MCI test errors I saw on test.winehq: the system "fg_winxp-ie8" that yields 1 error instead of 18: mci.c:391: Test failed: not enough time elapsed 80ms -- a flakey test. And on the ME systems: mci.c:603: Test failed: mci re-auto-open status mode returned error: 275 mci.c:612: Test failed: mci auto-open pause returned error: 275 Luckily, there are a few machines that pass all tests.