Am 18.04.2010 um 04:57 schrieb Dan Kegel:
At the moment, my plan is to simply dedicate a machine and script a daily run of a couple benchmarks with +fps, and upload the resulting framerates to a web page so we can notice if somebody checks in something that slows them down.
- Dan
I still have my cxtest scripts and php hackerish that runs a bunch of benchmarks, and I'm still running them. Unfortunately this stuff is a pain to maintain, for various reasons.
I'm testing 3dmark2k, 3dmark2k1, ut2004, halflife2 and team fortress 2. I am also running the Wine tests. The big requirement for the benchmark games is that I have a way to automatically read the results from the games. Running with +fps and collecting the output is imprecise IMO.
The main problems are:
*) Maintainance. The tf2 timedemo format changes every now and then, and similar issues *) The way I wrote the code. Very hacky. *) The test machines. I could use a server farm, but I don't have any. The boxes are standing in my living room. *) System updates(ubuntu) need to be installed and can affect performance. *) Unreliable apps. The steam login fails every now and then, requiring manual intervention.