http://kegel.com/wine/yagmarkdata/ now has data from 3dmark2000, 2001, and 06 running on a beefy i7 960 with an equally beefy Nvidia GTX 295 with either Windows 7 or Linux+Wine (all tests run at 1024x768).
Here are the results of one run for each plucked at random (sorry, no breakdown of 3dmark06 results yet):
Win 7 Wine units 3dmark2000_3DMark_Result: 39587 26278 3Dmarks 3dmark2000_CPU_Speed: 1269 1276 CPU_3Dmarks 3dmark2000_Game_1_Helicopter_Low_Detail: 689.7 415.2 FPS 3dmark2000_Game_1_Helicopter_Medium_Detail: 526.8 319.4 FPS 3dmark2000_Game_1_Helicopter_High_Detail: 412.1 260.1 FPS 3dmark2000_Game_2_Adventure_Low_Detail: 981.2 648.4 FPS 3dmark2000_Game_2_Adventure_Medium_Detail: 429.0 334.9 FPS 3dmark2000_Game_2_Adventure_High_Detail: 260.2 211.8 FPS
3dmark2001_3DMark_Score 46297 28194 3DMarks 3dmark2001_Game_1_Car_Chase_Low_Detail 551.3 314.6 fps 3dmark2001_Game_1_Car_Chase_High_Detail 111.5 133.0 fps 3dmark2001_Game_2_Dragothic_Low_Detail 678.0 462.2 fps 3dmark2001_Game_2_Dragothic_High_Detail 399.6 184.2 fps 3dmark2001_Game_3_Lobby_Low_Detail 602.7 371.5 fps 3dmark2001_Game_3_Lobby_High_Detail 294.0 135.5 fps 3dmark2001_Game_4_Nature 593.9 382.9 fps
3DMark06_result: 21519 6038 3dmarks
This might not be all that useful on its own, but once I have daily automatic runs going, it ought to be helpful in looking for performance regressions.
These results are probably only accurate to +- 5%. Later on I'll do multiple runs and compute the median to reduce the noise level.
Notes:
On really big screens, at least under wine, 3dmark2000 seems to lose its mind and put up a warning dialog behind the main window (thus hanging the test). I had to set the screen size down to something closer to 1024x768 for it to calm down and run the benchmark.
3dmark2000 and 3dmark2001 need to be run in xp compatibility mode on Windows 7, or they complain of insufficient memory.
Again, this was done by running http://code.google.com/p/winezeug/source/browse/trunk/yagmark - Dan
http://kegel.com/wine/yagmarkdata now has data from five different benchmarks: 3dmark{2000, 2001, 06} and heaven2_{opengl, d3d9}, and running on a semi-whimpy e8400 dual core box with an nvidia gt 220 card, on both Vista and Ubuntu+Wine.
First, the good news: the OpenGL version of the Heaven benchmark achieved 99% of the expected framerate on Wine, not bad, and it looks good, too. The 3dmark* demos look good in general.
And now the bad news: in general, Wine's D3D version achieves only half to three-quarters the performance of Vista's. The Heaven D3D benchmark doesn't look right in quite a few ways (and one regression is very recent), requires more video ram than on Windows, and hangs at the end. 3dmark2001 has a strange water problem in the nature test at 30 seconds. 3dmark06 lacks shadows in the firefly forest. Related bugs: http://bugs.winehq.org/show_bug.cgi?id=5776 http://bugs.winehq.org/show_bug.cgi?id=20777 http://bugs.winehq.org/show_bug.cgi?id=22404 http://bugs.winehq.org/show_bug.cgi?id=22614 http://bugs.winehq.org/show_bug.cgi?id=22623
Yagmark is now in fairly good shape, please try it yourself; see http://wiki.winehq.org/yagmark
The file http://kegel.com/wine/yagmarkdata/vista-vs-wine-1.1.44-19.txt shows how wine does vs. vista. Here's an excerpt (values given are medians from six to ten runs):
From: dank@kegel.com; CPU: E8400; GPU: GT220 benchmark_variable Vista wine-1.1.44-19 ratio 3dmark06_3DMark_Score 4868.00 3385.00 0.70 3dmark06_CPU1_Red_Valley 0.86 0.86 0.99 3dmark06_CPU_Score 2740.00 2705.00 0.99 3dmark06_GT1_Return_To_Proxycon 17.43 13.28 0.76 3dmark06_GT2_Firefly_Forest 14.35 10.64 0.74 3dmark06_HDR1_Canyon_Flight 16.80 9.73 0.58 3dmark06_HDR2_Deep_Freeze 19.19 11.35 0.59 3dmark06_HDR_SM3_0_Score 1800.00 1053.00 0.58 3dmark06_SM2_0_Score 1906.00 1433.00 0.75 3dmark2000_3DMark_Result 30262.00 17197.00 0.57 3dmark2000_CPU_Speed 1227.00 1082.00 0.88 3dmark2000_Game_1_Helicopter_High 283.60 151.60 0.53 3dmark2000_Game_2_Adventure_High 215.10 142.20 0.66 3dmark2001_3DMark_Score 28289.00 15634.00 0.55 3dmark2001_Game_1_Car_Chase_High 78.80 38.20 0.48 3dmark2001_Game_2_Dragothic_High 321.30 159.10 0.50 3dmark2001_Game_3_Lobby_High 231.10 109.60 0.47 3dmark2001_Game_4_Nature 114.00 106.70 0.94 heaven2_d3d9_FPS 18.38 9.30 0.51 heaven2_d3d9_Min_FPS 9.73 5.74 0.59 heaven2_d3d9_Scores 462.92 234.37 0.51 heaven2_gl_FPS 16.20 16.12 0.99 heaven2_gl_Min_FPS 10.16 9.33 0.92 heaven2_gl_Scores 408.19 406.08 0.99
- Dan
On Sun, May 9, 2010 at 4:14 PM, Dan Kegel dank@kegel.com wrote:
in general, Wine's D3D version achieves only half to three-quarters the performance of Vista's.
I just tried 'winetricks glsl-disable' on heaven2_d3d9. It sped it up about 8% (to 16.0 fps), but added some fun problems (e.g. scenes 7, 10, and 12 have a black sky).
On Mon, May 10, 2010 at 8:46 AM, Dan Kegel dank@kegel.com wrote:
On Sun, May 9, 2010 at 4:14 PM, Dan Kegel dank@kegel.com wrote:
in general, Wine's D3D version achieves only half to three-quarters the
performance
of Vista's.
I just tried 'winetricks glsl-disable' on heaven2_d3d9. It sped it up about 8% (to 16.0 fps), but added some fun problems (e.g. scenes 7, 10, and 12 have a black sky).
Hello Dan,
May I ask why no 3Dmark 03 or 05 benchmark results? Sorry if I missed a previous answer to this question.
Tom
On Mon, May 10, 2010 at 12:49 AM, Tom Wickline twickline@gmail.com wrote:
May I ask why no 3Dmark 03 or 05 benchmark results?
I'm trying to take care of Henri and Stefan's wishes/needs:
http://www.winehq.org/pipermail/wine-devel/2010-April/083083.html http://www.winehq.org/pipermail/wine-devel/2010-April/083091.html
and figured that before I added too many more benchmarks, I should get the automated daily report going. Adding wine3d3-on-windows tests, and ut2004, halflife2 and team fortress 2, and maybe Resident Evil 5 benchmark, are also on the to-do list. (Though that last one might require OCR; anyone know how to get FPS out of it into a text file on Windows?)
Patches to add support for more benchmarks would be welcome. (I hesitate to use Steam or Battle.net for any of these benchmarks because autoupdates and login problems make games hard to script.) I'm looking forward to the windows version of Phoronix Test Suite, will probably add it when it's ready. Are there any other benchmarks we should consider adding? - Dan
Am 10.05.2010 um 14:49 schrieb Dan Kegel:
(I hesitate to use Steam or Battle.net for any of these benchmarks because autoupdates and login problems make games hard to script.)
Steam is pretty nice in this regard. Everything you need can be controlled via Command line, e.g.
Steam.exe -login <user> <pass> -applaunch 220 -novid -console -window -w 1024 -h 768 -dxlevel 90 +timedemo mytimedemo
Everything after -applaunch 220 is a game argument. With Source engine games you can pass anything you can enter on the console via "+command". You can't make the game quit that way though, because if you pass "+timedemo xxx +quit" it will quit right after starting the timedemo - the timedemo command doesn't block.
For killing Steam I'm using "killall Steam.exe".
This page has the game appids: http://developer.valvesoftware.com/wiki/Steam_Application_IDs
However, this won't fix any issues caused by autoupdates. The updates might have an impact on performance, and especially with the newer games(TF2, L4D) the timedemo format changes, and your recorded demo becomes useless. This didn't happen recently with TF2 though, I am still using my timedemo from 2006
Also, there's an add popup after the game quits, which can be annoying if you run multiple tests without restarting Steam. It can be disabled via a Steam setting(or registry key)
On Mon, May 10, 2010 at 8:24 AM, Stefan Dösinger stefandoesinger@gmx.at wrote:
Steam is pretty nice in this regard. Everything you need can be controlled via Command line, e.g.
Steam.exe -login <user> <pass> -applaunch 220 -novid -console -window -w 1024 -h 768 -dxlevel 90 +timedemo mytimedemo
Everything after -applaunch 220 is a game argument. With Source engine games you can pass anything you can enter on the console via "+command". You can't make the game quit that way though, because if you pass "+timedemo xxx +quit" it will quit right after starting the timedemo - the timedemo command doesn't block.
I hear there's a +timedemoquit command....
Can you send me the timedemo files you use?
Also, there's an add popup after the game quits, which can be annoying if you run multiple tests without restarting Steam. It can be disabled via a Steam setting(or registry key)
Pray tell... - Dan
Am Donnerstag 13 Mai 2010 03:10:10 schrieb Dan Kegel:
I hear there's a +timedemoquit command....
Can you send me the timedemo files you use?
My timedemo is this one here: http://stud4.tuwien.ac.at/~e0526822/mydemo.dem.bz2
It is quite old, but still works. It is a rather long one, but I recorded it when Wine ran this game at ~25 fps. Nowadays it runs at 90-100 fps, so the demo runs rather fast.
On Fri, May 14, 2010 at 10:19 AM, Stefan Dösinger stefandoesinger@gmx.at wrote:
My timedemo is this one here: http://stud4.tuwien.ac.at/~e0526822/mydemo.dem.bz2
It is quite old, but still works. It is a rather long one, but I recorded it when Wine ran this game at ~25 fps. Nowadays it runs at 90-100 fps, so the demo runs rather fast.
It ran nicely (if gruesomely). But how do you get framerates out of this (on Windows)? http://www.digital-daily.com/video/hl2-benchmarking/ claims it shows them in the console, and saves them to the file hl2hl2source.csv, but neither of those seemed to happen.
Here's what I did:
cd ~/".wine/drive_c/Program Files/Steam" cp ~/mydemo.dem config wine steam.exe -login myuser mypass -applaunch 220 -novid -console -window -w 1024 -h 768 -dxlevel 90 +timedemoquit mydemo.dem
I also ran it with WINEDEBUG=+file and +timedemo, but there was no .csv opened ever :-( - Dan
On Fri, May 14, 2010 at 11:27 AM, Dan Kegel dank@kegel.com wrote:
It ran nicely (if gruesomely). But how do you get framerates out of this (on Windows)? http://www.digital-daily.com/video/hl2-benchmarking/ claims it shows them in the console, and saves them to the file hl2hl2source.csv,
http://www.techenclave.com/gaming/half-life-2-tweaks-cheats-mods-10247.html is more specific, it says
timedemo [demoname] - Plays the specified demo and reports performance information upon completion, including frames played, time taken, average FPS and FPS variability. Also records the information in a file called sourcebench.csv in your \Program Files\Valve\Steam\SteamApps[username]\half-life 2\hl2\ directory.
I see that directory, but there's no new file in it :-( - Dan
On Fri, May 14, 2010 at 11:32 AM, Dan Kegel dank@kegel.com wrote:
On Fri, May 14, 2010 at 11:27 AM, Dan Kegel dank@kegel.com wrote:
It ran nicely (if gruesomely). But how do you get framerates out of this (on Windows)? http://www.digital-daily.com/video/hl2-benchmarking/ claims it shows them in the console, and saves them to the file hl2hl2source.csv,
Aha!
wine steam.exe -login myuser mypass -applaunch 220 -novid -console -window -w 1024 -h 768 -dxlevel 90 -condebug +timedemoquit mydemo.dem
will probably do what I want!
Am Freitag 14 Mai 2010 20:32:06 schrieb Dan Kegel:
timedemo [demoname] - Plays the specified demo and reports performance information upon completion, including frames played, time taken, average FPS and FPS variability. Also records the information in a file called sourcebench.csv in your \Program Files\Valve\Steam\SteamApps[username]\half-life 2\hl2\ directory.
I see that directory, but there's no new file in it :-(
I see the sourcebench.csv file in that directory... Not sure why it doesn't work for you.
On Mon, May 10, 2010 at 3:49 PM, Dan Kegel dank@kegel.com wrote:
Are there any other benchmarks we should consider adding?
Eve Online: - there are many Eve players who depend on wine, so making it run faster should make many happy - lots of graphics options (so many different aspects can be tested)
Downside is that it is an MMORPG, and I don't know how easy it can be scripted. Maybe some other Eve players here have more experience with that. There is an in-game FPS window which draws a graph, but I'm not sure about writing FPS to files. Also, there is both a live server (Singularity) and a test server (Tranquility).
Another problem is that wine-1.43 seems to give an error with d3d when starting up (unimplemented function d3dx9_36.dll.D3DXCreateEffectEx -- I did notice there is a recent patch in git that adds a stub). I should reinstall the game with an empty prefix and test again. A few wine betas ago it was surely working, so if there are any regressions they will hopefully be fixed soon.
Octavian
On Mon, May 10, 2010 at 5:49 AM, Dan Kegel dank@kegel.com wrote:
I'm trying to take care of Henri and Stefan's wishes/needs:
http://www.winehq.org/pipermail/wine-devel/2010-April/083083.html http://www.winehq.org/pipermail/wine-devel/2010-April/083091.html
and figured that before I added too many more benchmarks, I should get the automated daily report going. Adding wine3d3-on-windows tests, and ut2004, halflife2 and team fortress 2, and maybe Resident Evil 5 benchmark, are also on the to-do list.
I've added orangebox to wisotool (what a pain!), and will use the notes in http://www.digital-daily.com/video/hl2-benchmarking/ to add hl2 to yagmark. (Though I'm still apprehensive about how much babysitting it will need.) - Dan
Am 10.05.2010 02:46, schrieb Dan Kegel:
On Sun, May 9, 2010 at 4:14 PM, Dan Kegeldank@kegel.com wrote:
in general, Wine's D3D version achieves only half to three-quarters the performance of Vista's.
I just tried 'winetricks glsl-disable' on heaven2_d3d9. It sped it up about 8% (to 16.0 fps), but added some fun problems (e.g. scenes 7, 10, and 12 have a black sky).
Sceenes 7, 10 and 12 are totally black here on wine on opengl. But the linux native client has the same problem. So I guess there opengl engine is lacking some futures.
2010/5/13 Rico Schüller kgbricola@web.de:
I just tried 'winetricks glsl-disable' on heaven2_d3d9. It sped it up about 8% (to 16.0 fps), but added some fun problems (e.g. scenes 7, 10, and 12 have a black sky).
Sceenes 7, 10 and 12 are totally black here on wine on opengl. But the linux native client has the same problem. So I guess there opengl engine is lacking some futures.
That's interesting. What graphics hardware + drivers are you using?
Am 13.05.2010 14:52, schrieb Dan Kegel:
2010/5/13 Rico Schüllerkgbricola@web.de:
I just tried 'winetricks glsl-disable' on heaven2_d3d9. It sped it up about 8% (to 16.0 fps), but added some fun problems (e.g. scenes 7, 10, and 12 have a black sky).
Sceenes 7, 10 and 12 are totally black here on wine on opengl. But the linux native client has the same problem. So I guess there opengl engine is lacking some futures.
That's interesting. What graphics hardware + drivers are you using?
Geforce 8800 GTS, 195.36.15
It could also be a bad driver. Could you see all scenes correctly using Heaven_gl.bat ? The easiest way to detect this is to run two copys, one with d3d9 and one with opengl. This way one could see, when a scene is not drawn. Otherwise you could think that the black screen is only the scene switching.
-
Dan Kegel -
Octavian Voicu -
Rico Schüller -
Stefan Dösinger -
Tom Wickline