Re: [ANN] Conformance testing campaign

[Dimitrie O. Paun]

On August 29, 2003 05:50 pm, Ferenc Wagner wrote:
> Please go to http://afavant.elte.hu/~wferi/wine for details.

Hi Feri,

This stuff looks great -- maybe it's time we integrate
it into WineHQ...

A few comments on the current state of affairs:
  -- for the ME case, how can we have have some
     results (up to kernel32.dll:codepage) and then
     have no results? Doesn't that mean that they failed?

-- maybe we need a small legend at the top of the
     page, below "Main summary for build..."
     Something like:

Legend:
     <span class="pass">Tests pass in all reports</span>
     <span class="fail">Tests fail in all reports</span>
     <span class="mixed">Tests fail in some reports</span>

-- It would be nice to make the tests name links to
     the test file. For example, assuming these tests
     results were for the 20030813 build of wine,
     kernel32:alloc would be a link to:

http://cvs.winehq.com/cvsweb/wine/dlls/kernel/tests/alloc.c?only_with_tag=Wi...

This is trick though, as it is not clear how to figure
     this thing out. Maybe the solution is to have a mode
     for tests where all they do they print their filename,
     like so:
     kernel32.alloc:dlls/kernel/tests/alloc.c

-- Nice touch for the column span in 
     "Unit tests for ... where not compiled...". Cool!

-- How do you assign the name to different reports
     for the same OS? Like in
     Win98	Win98 joel	Win98 kevin
     Does the script do this automatically, or it requires
     manual intervention. It would be nice to be automatic,
     so when we install it at WineHQ, "it just runs" :)

-- Also, in the "XXX differences" section, shouldn't we 
     have the exact version & ServicePack displayed between
     the OS name and the reporter link, as we are dealing
     with only one instance?

-- A few links at the top of the page would be nice:
    1. To the main Testing page, something like
    	/site/testing
    2. To the next/prev set of results, like so
    	/?tests="20030813"&dir="next"
    For this we need a bit of help from Jer to implement,
    it's not possible to implement it properly as a
    static link in the page.

Anyway, all this are minor nits, and can be implemented
later -- I think the important stuff is to get this stuff
integrated in the WineHQ. We also need a "Testing" page,
but that's another story :)

Good stuff!