I second Vit's recommendation. We're using Autohotkey scripts in our test framework http://code.google.com/p/yawt/ and I've been meaning to work with Vit to get his scripts incorporated into our framework. (We've been distracted getting the next release of Picasa out, sorry.)
I had a look at autohotkey, and after some fiddling I managed to automatically load a predefined 3DMark2000 test, run it and write the results to the disk.
The whole thing has some rough edges, like hotkeys not working sometimes from autohotkey(but they do if entered manually) which I have to overcome by hardcoding button click coordinates. Also I found myself unable to send a backslash(for a path), but overall I think its a useable solution.
Appart of controlling the apps we'll need some programs to start the application scripts, extract the results and do something with them. That could be some bash or perl scripts.
We also need some database to collect the results. This could be something webserver based with some simple SQL database, or just another set of perl scripts on a server. Here I'd like to look how much cxtest infrastructure we can reuse. Any volunteers for writing such a thing?