--- Alexandre Julliard julliard@winehq.com wrote:
Andriy Palamarchuk apa3a@yahoo.com writes:
I frankly don't see the need of a complex test harness to launch tests and print reports. Maybe when we have 1000 tests we will want some of that complexity, but for now it's only a waste of time.
Almost all complexity you are talking about is already implemented for us. Usage of the framework is very simple, do not require from test writers anything. They are only required to correctly use Test::Simple (or Test::More). They don't need to remember about Test::Harness.
There are reasons to use Test::Harness: 1) control of TODO tests - I really want to use this feature. 2) control of SKIP tests - very useful for Wine-specific tests, choosing behavior, depending on Windows versions, etc. I need this feature too. 3) we already need to manage the test output. I'd estimate number of checks for my existing SystemParametersInfo unit test as: 25 (number of implemented actions) * 10 (minimal number of checks for each action) = 250 - 350 tests We'll definitely have huge number of tests. Why not pick up scaleable approach from very beginning?
Let's not debate licenses again. Anything that is included in the Wine distribution has to be under the Wine license. If you want a GPLed test suite you'll have to distribute it separately from Wine; but I think that would be a mistake.
IMHO you can distribute them together if these are 2 separate applications, right? I don't have big preferences about the license. Wine license will work fine.
We spent a lot of our valuable time and resources on the issue. Lets finalize the decisions and start to work.
Suggest decisions from the discussion: 1) unit tests are very important for the project 2) mixed C/Perl environment will be used for the tests development. Choosing the tool is a matter of personal preferences.
Comment: Alexandre, basing on the discussion above I think we both came to this. Do I understand you correctly? TODO: - create support in the build process
3) Test::Harness will be used to compile report for test batches
Comment - TODO: - fix problems in using Test::Harness (and some other modules, BTW) with winetest Perl framework - create simple framework for C to print report in format, suitable for Test::Harness. (I already started to work on this.) - change Test::Harness to run not only Perl scripts, but also other types of executables (can do this)
4) The unit test will be a separate application
Comment: Alexandre, we explicitely did not agree on this decision yet. You preferred to have unit tests spreaded over the Wine directory tree. The main argument for this was possibility of running subsets of test. I suggest to change Test::Harness (or create new module, based on it) to use tree-like tests organization. If you need to test module wine/dll/foo, you request to run tests hierarchy "wine/dll/foo".
TODO: - customize Test::Harness to use tree-like test suits structure (I can do this).
5) Existing Wine license will be used for the Unit test application
Please, let me know if I missed something or your point of view is not reflected.
Looking forward for your comments. Andriy Palamarchuk
__________________________________________________ Do You Yahoo!? Send FREE video emails in Yahoo! Mail! http://promo.yahoo.com/videomail/