Hello Stefan,
following is specification proposal for first few iterations. It is more focused on functionality than technical details. Feel free to comment.
1st iteration =============
* We take your 3dMark test, adjust it a bit and make it run and submit results regularly on one of our testing machines. This involves adding support for user defined results interpretation into CxTest framework. That means test creator says in test what data put into tables/graphs generated by CxTest server. Also some other things, like ability to turn off taking of screenshots.
* We add Direct3d tests evaluation here: http://www.cxtest.org/evaluation Basically, you select time frame and get table similar to attached one. Please comment.
* After few iterations, when we are satisfied how previous point is implemented, we help you to regularly run 3dMark test on your machine.
2nd iteration =============
* We add graph support on Direct3d evaluation page
* We add 2 more benchmark tests
3rd iteration =============
* We make it easy for end user to run our set of tests. They will be probably part of cxtest.sh standard tests (Wine "make test", WordViewer, ExcelViewer, PptViewer, Picasa, Direct3D tests - optional, user will be asked whether he has 3D acceleration and wants to run them, since they make computer unusable for the duration of the test)
Martin