On 11/09/2009 07:25 PM, Joerg-Cyril.Hoehle@t-systems.com wrote:
Reece,
I think we reached agreement on: a) Tests should "usually" succeed.
I didn't read that in any of the emails. Tests should always succeed. the broken() and skip() will help in achieving that.
The point is, if the machine has a broken soundcard (or in Wine does not have an available sound driver, or is broken due to PulseAudio on Ubuntu), the tests should still pass (or be skipped) as is appropriate.
This seems wrong (in general) to me, even though Paul somehow expresses an opinion similar to yours. I say you cannot write a program meant to run in a completely unknown environment, doing comprehensive tests and not stumbling upon strange behaviour.
I'm now busy for example with the eventlog stuff. I create loads of tests and run them on my pretty clean boxes (W95 up to Win7). Only when they pass on all boxes I sent the patches. If test.winehq.org shows failures for these new tests they have to be dealt with.
I agree that not everybody has this multitude of test boxes but there are always people around willing to test new tests on their Windows boxes (real and/or virtual).
Regards, Jörg Höhle.