Dimi, I see that you've added me as a "volunteer" for running the conformance tests on NT. Gee, thanks :).
In any case, I don't mind doing this. However, I have a question : what should I do with the test results? Posting them to wine-devel every time a test changes seems somewhat high-bandwidth...
What we (i.e., the testers) could do is to keep track of what changes between successive test runs, I guess, and we could just post a summary of the changes.
This brings up another point: these test results should be put up somewhere on the Web for everyone to see. Might I suggest putting them up on the Wine Janitorial Projects page? For every platform, just put in a link to "Most recent test results", say.
One last thing: is there any way to be automatically notified whenever the available test suite changes?
On December 11, 2002 12:21 pm, Luke Stras wrote:
Dimi, I see that you've added me as a "volunteer" for running the conformance tests on NT. Gee, thanks :).
You're welcome! :)
In any case, I don't mind doing this. However, I have a question : what should I do with the test results? Posting them to wine-devel every time a test changes seems somewhat high-bandwidth...
Well, theoretically these tests should not fail. So once we fix them, you should post a message to wine-devel if and only if you notice a failure.
What we (i.e., the testers) could do is to keep track of what changes between successive test runs, I guess, and we could just post a summary of the changes.
Yes, that would be useful to track progress until we get all tests fixed. diff(1) is your friend. But once the tests are fixed, you only need to post if you notice any failure, as explained above.
This brings up another point: these test results should be put up somewhere on the Web for everyone to see. Might I suggest putting them up on the Wine Janitorial Projects page? For every platform, just put in a link to "Most recent test results", say.
I've added a link there to the binaries posted by Francois. Would you prefer something else?
One last thing: is there any way to be automatically notified whenever the available test suite changes?
I suggested it, but someone has to 1. Write a script that build the tests, creates an archive, compares the result with the previous build, and if different, emails the guys on the testing list 2. We need a volunteer that runs the script from time to time (say sets up a cron job on a box that does cd "$WINEROOT" && cvs up && tools/update-tests or something like this)
I've also included on the 'Fun Projects' page an idea for a nice test harness (a la JUnit). Check it out -- perfect project for a newbie with some Win32 programming experience.
-
Dimitrie O. Paun -
Luke Stras