--- Alexandre Julliard julliard@winehq.com wrote:
Andriy Palamarchuk apa3a@yahoo.com writes:
[...]
But adapting the framework to do what we want is IMO more work than simply reimplementing from scratch the few features that we actually need. We don't really gain anything by making the effort of reusing all that code, since we don't need most of it.
Could you, please, list the additional features we need? I'll try to estimate amount of work necessary to implement them in Test::Harness.
Do you want to use architecture, completely different from Test::Harness + Test::Simple modules or you only want to replace Test::Harness?
Existing architecture: Individual test scripts (executables) are very simple (use Test::Simple). They only print messages like "ok 4 - action SPI_FOO # TODO not implemented" for each check. Test::Harness module parses output, has logic to analyze test results, creates overall report and details of failures, including crashes.
I don't want to know about the details of the 250 individual checks.
You got an impression that Test::Harness does not report individual failures. Sorry, I gave too simple demo.
Example of Test::Harness output for a few failures:
test2.p.............NOK 2# Failed test (test2.pl at line 8) # got: '1' # expected: '0' test2.p.............NOK 3# Failed test (test2.pl at line 9) # got: '1' # expected: '0' test2.p.............NOK 4# Failed test (test2.pl at line 10) test2.p.............ok 8/8# Looks like you failed 3 tests of 8.
[... summary report output is skipped ...]
Does this output look closer to the one you want? Let me know if you need any other information.
- mixed C/Perl environment will be used for the
tests
development. Choosing the tool is a matter of
personal
preferences.
I don't think I agree. For me the value of Perl is in that it makes it trivial to take the suite over the Windows; but if half the tests are in C we lose this advantage, and then we might as well do everything in C.
Sorry, misinterpreted your statement that threads won't be used for tests in Perl. Could you give your vision when C is used?
What I want is a make-like system that keeps track of which tests have been run, which ones need to be re-run because they have been modified etc.
This usage of make is fine with me. I just want to separate unit tests from main code and have centralized control. You still can call subset of unit tests from the build process.
You cannot put the whole test suite in a single application, you need to split things up some way. A decent test suite will probably be several times the size of the code it is testing; you don't want that in a single application.
We were not careful about the terms. Yes, the tests will be bunch of the executables. I assumed these executables to be parts of one test application. Completely agree with you.
You preferred to have unit tests spreaded over the Wine directory tree. The main argument for this was possibility of running
subsets
of test.
No, the argument is modularity.
I'm all for modular unit tests and there are a few ways to divide the tests between modules.
Then when you change something in kernel32 you can change the test that is right next to it, run make test in the kernel32 directory and have it re-run the relevant tests, and then do cvs diff dlls/kernel32 and get a complete diff including the code and the test changes.
Agree, this is not so convenient for separate test application.
Separate unit tests application has its own advantages: 1) Separate distributions are used for Wine and the test applications. The only case when we want them together - Wine development under *nix. In all other cases we need only one of them. 2) The unit tests will be mostly developed under Windows. The unit tests build process has extra Windows compatibility requirements. 3) Having unit tests as a separate application we create possibilities to collaborate with other W32 implementation OS projects (ODIN comes to mind).
All the tasks we mentioned can be implemented with any approach. I believe the tasks you described will be not much more difficult with my approach, especially if directory tree has parallel structure.
Andriy Palamarchuk
__________________________________________________ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/