- Ability to regularly benchmark certain parts of game (and thus Wine,
which is involved) in clean environment and submit raw data results to server
- On the server, process submitted data and present them via web in user
friendly form; support for pointing to regressions (and finding regression patches) would be nice
Did I miss some important aspect? Feel free to comment.
Yes, thats essentially what we need. I've been setting up a quick and dirty test site myself, with a few lines of php and gnuplot:
http://84.112.174.163/~stefan/laptop/3dmark2000/results.php
The whole setup is highly inflexible, doesn't contain any real test data yet, just quick and dirty(~100 lines of php and ~70 lines of gnuplot script).