Francois Gouget fgouget@free.fr writes:
I think what we need with this is a couple of guidelines and documentation for potential test writers, and maybe a couple of extensions. The following is half a proposed documentation that we could put in the Wine Developper Guide, and half a proposed specification for some possible extensions. As usual, comments and suggestions are welcome.
Great job!
- it should be easy to correlate with the source of the test. For
instance if a check fails, it would be a good idea to print a message that can easily be grepped in the source code, or even the line number for that check. But don't print line numbers for success messages, they will change whenever someone changes the test and would require an update to the reference files..
IMO the test should not be printing successes or failures at all. If it can determine whether some result is OK or not, it should simply do an assert on the result. Printing things should be only for the cases where checking for failure is too complicated, and so we need to rely on the output comparison to detect failures.
Otherwise this file is either called:
- 'xxx.ref'
- or 'xxx.win95' or 'xxx.win98' ... if the output depends on the
Windows version being emulated. The winever-specific file takes precedence over the '.ref' file, and the '.ref' file, which should exist, serves as a fallback.
No, there should always be a single .ref file IMO. Version checks should be done inside the test itself to make sure the output is always the same.
Each test should contain a section that looks something like:
# @START_TEST_COVERAGE@ # kernel32.CreateProcess # kernel32.GetCommandLineA # kernel32.GetCommandLineW # __getmainargs # __p__argc # __p_argv # __p_wargv # @END_TEST_COVERAGE@
This is already part of the Perl framework, you have to explicitly declare the functions you use. So we don't want to duplicate the information.
- TEST_WINVER This contains the value of the '-winver' Wine argument, or the
Windows version if the test is being run in Windows. We should mandate the use of specific values of winver so that test don't have to recognize all the synonyms of win2000 (nt2k...), etc. (Do we need to distinguish between Windows and Wine?)
The test should use GetVersion() and friends IMO, no need for a separate variable.
- and add two corresponding targets: 'make cui-tests' runs only those
tests that do not pop up windows, and 'make gui-tests' runs only those tests that do pop up windows
- 'make tests' would be 'tests: cui-tests gui-tests'
I don't think this complexity is necessary. You can always redirect the display if the windows annoy you. And tests should try to keep the windows hidden as much as possible.
What is needed most is a two sample tests:
- one simple console based test
- another one involving some GUI stuff
I have attached two sample Perl scripts that were written some time ago by John Sturtz and myself. One is testing the atom functions and the other is creating a window. They should probably be simplified a bit to serve as documentation samples.