Hi Tom,
On Mon, 2 Sep 2002, tom wrote:
Paul Millar wrote:
On Sun, 1 Sep 2002, tom wrote:
I have a pre-1 hacked together and request any feed back that you can give.
I'm doing rolling CVS regression/conformance testing and reporting back whenever something breaks. How about adding an extra line under the "Wine Regression Test Suite"?
Hi Paul,
Is this ** Part ** of the Regression Test Suite or a different category to add ?
It could probably be argued either way ... :) In the limited meaning of Regression Test Suite, then the answer is no.
I am not a developer so please look over me if this is a dumb question ... If it is part of the Regression Test Suite I will just add you in as a worker with CodeWeavers .
Well, I don't claim to have written anything in the Regression Test Suite (only one completely trivial patch), so I'd rather my name wasn't in the "workers" section.
If it is a Different category I think it shold be like this
Rolling cvs testing Documentation = workers = Paul Millar % = 10%
Hmmm, yes. On the whole, I think this is better. You can put me down for Documentation too. I'm not sure what (if any) documentation is really needed, but I think there's a reasonable amount at: http://www.astro.gla.ac.uk/users/paulm/WRT/ and http://www.astro.gla.ac.uk/users/paulm/WRT/help.php
Its difficult to gauge a percent-completeness for the rolling regression testing. I'm quite happy with the functionality as it stands now (although I've got a few ideas for improvements). Its been quite stable (apart from some problems with thread deadlocks) for quite a while now, so I'd say its probably around 90% or so.
Is the ( Regression Test Suite ) a program to test the Regression % or the % of the Regression ? As one is a app the other is a % If it is a app do you know the current % done ?
I don't know if this answers your question, but the Regression Test Suite is a series of explicit statements of what is correct behaviour of the Windows API. Calling function Foo() with parameter "Bar" should return "Baz" : that sort of thing. Information is gleaned from MS's official documentation and from direct testing of various Windows platforms. There's a little more info on the first URL above.
Ideally we want all the functions implemented by wine to precisely match the result from Windows under all circumstances (or at least all circumstances where it matters for a program). This behaviour should then be tested to make sure nothing breaks at a later date.
For calculating a percentage, the script calculates the number of APIs implemented, and the number of those functions that appears in at least one regression test. The idea is divide one by the other and you have a measure of what percent of wine is tested by the Regression Suite (hence how complete it is).
Unfortunate the estimate almost certainly an over-estimate. The calculated number of implemented API is via a crude grep. The estimate also assumes that, just because a function is mentioned in a regression test, the function should be counted as "tested". Both bias towards an increased percentage, so the 10.3% value should be taken with a generous pinch of salt :)
Cheers,
Paul.
---- Paul Millar