Alexandre Julliard wrote:
I don't really care about the output. My point is
that this output is
not necessary, a simple "assertion foo failed at
line xx" on failure
and no output at all on success would work just as
well. I've nothing
against such an output either, but I don't think it
justifies
introducing the complexity of reusing the Test
stuff.
I already created small library with api like in Test::Simple - functions ok, TODO, SKIP. The implementation can be easy changed to the behaviour you described. I also suggest for library to fail on successful TODO test. TODO tests will be conditional - they are TODO only under Wine, not on Windows.
Careful usage SKIP sections can easy cover differences between Windows platforms, using TODO - between different non-Microsoft Win32 implementations.
On other hand, SKIP sections were important for Test::Harness, which needed to control number of tests. For your goals conditional execution of the tests is sufficient.
Which approaches can you suggest to reach goals I believe are important:
1) bundling - are we going to have separate distributions - Wine with tests, tests only, Wine only? There are a lot of cases where only one of them is required.
2) development of the unit tests under Windows. Obviously, we don't need to have Wine itself when we work with unit tests on Windows. Plus, we need to create development environment, usable by Windows developers.
3) Organization of the unit tests in such way, so they can be used by other Win32 implementation projects. Conditional TODOs I suggested above will help to manage different TODO lists for different projects.
4) besides narrow-scope regression testing you suggested we need to run all the tests from time to time, even if it takes more than 1 hour to run the whole suite. The reasons for this are:
a) we need to run the tests on all Windows platforms to make sure the tests correctly document their behaviour
b) Wine should be reguraly tested by the whole testing suite. Some regressions will be caught in unit tests of the modules other than changed. In other cases changed module may not have tests at all, but will brake modules, dependent on it.
Thanks, Andriy Palamarchuk
__________________________________________________ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/