winetest: what next? - wine-devel

20 Apr 2004


      OK,
So assume we get the show rolling, tests will run on various boxes,
results will be send back to WineHQ, nice colorful pages will be
generated, and a beautiful index page will put all these together,
being linked from the Status (http://www.winehq.org/site/status) page.
Are we done? I think not. This is a very important piece of 
infrastructure, and it's buried rather deep into the bowels
of WineHQ. Moreover, it's not easy to monitor the changes in
testing status.
As such, I would like to have a "Weekly testing update" posted
to in the News section, with stuff like: how many tests were
added, how many failed/successed, etc. since last week. Maybe
a list of DLLs where changes were observed, and so on.
To get these statistics (and possibly other interesing bits)
in the future, I'm thinking at storing all these test results
in the database. NB: I'm not proposing that we change the
current processing of dissect & gather, it is very nice, and
it's a big bonus that it works without a database. All I'm
saying is that we can augment dissect to also save stuff to
the database.
For the database, we need two simple tables: 
    wine_test_releases, and wine_test_results.
wine_test_releases:
    build_id		VARCHAR(20) NOT NULL
    cvs_ts		VARCHAR(20)
    cvs_tag		VARCHAR(20)
    url			VARCHAR(256) NOT NULL
    publish_ts		TIMESTAMP NOT NULL
This table will be populated by Brian's script, when a new
release is publised. One of cvs_ts or cvs_tag will be NOT NULL,
and will specify how to get to the exact source used for the
build. Nightly builds will use cvs_ts (cvs -D <cvs_ts>), while
official releases done by Alexander will use cvs_tag 
(cvs -r <cvs_tag>). Additional metadata may be required here,
suggestions welcome.
wine_test_results:
    build_id		VARCHAR(20) NOT NULL
    tester_id		VARCHAR(20)
    test_name		VARCHAR(20) NOT NULL
    subtest_name	VARCHAR(40) NOT NULL
    file_name		VARCHAR(128) NOT NULL
    line_number		INTEGER NOT NULL
    result		INTEGER         // 0 = success, 1 = timeout, etc
    windows_version	VARCHAR (15)    // 4.3.1233
I've probably missed something, so feel free to add to this.
Comments, flames?
--  
Dimi.