Benchmark's on the Wiki
Hello, I put the Benchmark results that I posted to wine-devel back in April on the Wiki. http://wiki.winehq.org/BenchMark I would appreciate any constitutive criticism to improve this page. Tom
Tom Wickline wrote:
Hello,
I put the Benchmark results that I posted to wine-devel back in April on the Wiki. http://wiki.winehq.org/BenchMark
I would appreciate any constitutive criticism to improve this page.
Tom
.
I'd be interested to see how 3DMark05 runs. Do Oliver's patches support it yet? --Mitchell -- I find that a great part of the information I have was acquired by looking up something and finding something else on the way. -- Franklin P. Adams
On 7/26/05, Mitchell Mebane <mmebane(a)ev1.net> wrote:
I'd be interested to see how 3DMark05 runs. Do Oliver's patches support it yet?
Back in April 3DMark05 would install and load but there was a lack of Pixel Shader 2.0 support, so none of the test would run. http://www.futuremark.com/products/3dmark05/?minreq A quick look at : http://directxwine.sourceforge.net/ "pixel shaders should be implemented in the next release or two depending on the amount of regression and other bugs that popup." Oliver would you have a ETA for Pixel Shader 2/3 ? Tom
--Mitchell
--- Tom Wickline <twickline(a)gmail.com> wrote:
On 7/26/05, Mitchell Mebane <mmebane(a)ev1.net> wrote:
I'd be interested to see how 3DMark05 runs. Do Oliver's patches support it yet?
Back in April 3DMark05 would install and load but there was a lack of Pixel Shader 2.0 support, so none of the test would run.
http://www.futuremark.com/products/3dmark05/?minreq
A quick look at : http://directxwine.sourceforge.net/ "pixel shaders should be implemented in the next release or two depending on the amount of regression and other bugs that popup."
Oliver would you have a ETA for Pixel Shader 2/3 ?
I hope to get vertexshaders into winecvs by Friday this week, it's more or less the same as vertexshader support in d3d8 but with a load of parser fixes (it can now parse V2 and 3 shaders). Pixelshaders are more or less identical to vertexshaders so they should follow soon afterwards. To properly support version 2 and 3 shaders we need to use GL shader language instead of vertex and fragment program, it expect it to take a couple of weeks to get a cross compiler up and running. Blends also needs to be implemented using shaders to efficiently use VBO and the cards onboard memory for animated models. This affect more applications / games than V2 and 3 shaders so I'll try and develop it first.
Tom
--Mitchell
___________________________________________________________ Does your mail provider give you FREE antivirus protection? Get Yahoo! Mail http://uk.mail.yahoo.com
Tom Wickline wrote:
Hello,
I put the Benchmark results that I posted to wine-devel back in April on the Wiki. http://wiki.winehq.org/BenchMark
I would appreciate any constitutive criticism to improve this page.
I think it would be easier to read if the wine and xp results were side-to-side in a table? -- Anssi Hannula
Anssi Hannula wrote:
Tom Wickline wrote:
I put the Benchmark results that I posted to wine-devel back in April on the Wiki. http://wiki.winehq.org/BenchMark
I would appreciate any constitutive criticism to improve this page.
I think it would be easier to read if the wine and xp results were side-to-side in a table?
So Codeweavers' email/list server delayed my email for 11 hours so that Paul would beat me to it and propose the same idea before me? unfair :P -- Anssi Hannula
"Anssi Hannula" <anssi.hannula(a)gmail.com> wrote:
So Codeweavers' email/list server delayed my email for 11 hours so that Paul would beat me to it and propose the same idea before me? unfair :P
Are you a subscriber of the mailing list with the address you were writing from? -- Dmitry.
Dmitry Timoshkov wrote:
"Anssi Hannula" <anssi.hannula(a)gmail.com> wrote:
So Codeweavers' email/list server delayed my email for 11 hours so that Paul would beat me to it and propose the same idea before me? unfair :P
Are you a subscriber of the mailing list with the address you were writing from?
Oops, I thought I switched the subscriber address to this list when I switched others. Ok, that explains it :) -- Anssi Hannula
On 7/27/05, Anssi Hannula <anssi.hannula(a)gmail.com> wrote:
Anssi Hannula wrote: So Codeweavers' email/list server delayed my email for 11 hours so that Paul would beat me to it and propose the same idea before me? unfair :P
:-) Hello Anssi, Is this what you guys had in mind? http://wiki.winehq.org/BenchMark And just add a third column with +/- %'s Any other Recommendations? Tom
-- Anssi Hannula
On Thu, Jul 28, 2005 at 04:29:53AM -0400, Tom Wickline wrote:
And just add a third column with +/- %'s
can the test runs be automated like the winetests? and so have them updated automatically with different machines? or is there a way to not only test "functionality" with tests but also "speed"? i guess this would be nice to find bottlenecks. (maybe this is already done - if so excuse my ignorance ;)) -- cu
hi, Christoph Frick wrote on 07/28/2005 11:55 AM:
or is there a way to not only test "functionality" with tests but also "speed"? i guess this would be nice to find bottlenecks.
(maybe this is already done - if so excuse my ignorance ;))
has anyone ever tried to use grpof or any other profiling tool to analyze, which code eats most cpu time (=bottlenecks). normally, if you develop something, you first go for functionality and after that for speed. it's not good for wine to hack for speed (like cedega does, as far as I know) and recognize later, that it was the wrong decision, because of incompatibilities or incompleteness. (my professor would say "..and now we can calculate bullshit in a nanosecond"). nevertheless, it would not be wrong to have some clearly arranged detailed time profiling of wine, when started with several different applications. holger.
On Thu, Jul 28, 2005 at 05:34:08PM +0200, Holger Dell wrote:
or is there a way to not only test "functionality" with tests but also "speed"? i guess this would be nice to find bottlenecks. has anyone ever tried to use grpof or any other profiling tool to analyze, which code eats most cpu time (=bottlenecks).
i would consider gprof a tool to optimize an application - but its a little hard to profile an API with it. different apps will produces different results and so using gprof with certain apps (mainstream) results in _exaclty_ what you dont want (what transgaming does?). thinking about it - also using the benchmarks as a mark is the same (isnt that what all the driver developers do?). but having winetests, that also do certain things (lets say application "patterns") and then time them is IMHO a Good Thing - it can be coded into the winetest framework, which already runs the tests on real windows and on wine - then also time it and compare it. especially in the directx area this would be good to compare to windows (even if its sometimes simply not fair - if e.g. sound mixing is done in hardware under windows and in the cpu on wine). -- cu
On 7/28/05, Holger Dell <mail(a)holger-dell.de> wrote:
has anyone ever tried to use grpof or any other profiling tool to analyze, which code eats most cpu time (=bottlenecks). normally, if you develop something, you first go for functionality and after that for speed. it's not good for wine to hack for speed (like cedega does, as far as I know) and recognize later, that it was the wrong decision, because of incompatibilities or incompleteness.
Anyone here use Cedega and have the time to waist and benchmark it against Wine? Would a modern benchmark even run on there Warez? As im a advocate of OSS I don't use there offerings. Tom
holger.
hi, Tom Wickline wrote on 07/28/2005 10:29 AM:
Is this what you guys had in mind? http://wiki.winehq.org/BenchMark
I don't think so. The table format I had expected is: $(benchmark name) | $(wine result) | $(XP result) | $([any other interesting OS] result)
Any other Recommendations?
Mac OS X benchmarking results (where available) would be cool. holger.
Holger Dell wrote:
Tom Wickline wrote on 07/28/2005 10:29 AM:
Is this what you guys had in mind? http://wiki.winehq.org/BenchMark
I don't think so. The table format I had expected is:
$(benchmark name) | $(wine result) | $(XP result) | $([any other interesting OS] result)
Any other Recommendations?
Mac OS X benchmarking results (where available) would be cool. I do not think that MacOS X runs on his hardware. Using an other box for that would be like comparing apples with oranges.
bye michael -- Michael Stefaniuc Tel.: +49-711-96437-199 Sr. Network Engineer Fax.: +49-711-96437-111 Red Hat GmbH Email: mstefani(a)redhat.com Hauptstaetterstr. 58 http://www.redhat.de/ D-70178 Stuttgart
On 7/28/05, Michael Stefaniuc <mstefani(a)redhat.com> wrote:
Mac OS X benchmarking results (where available) would be cool. I do not think that MacOS X runs on his hardware. Using an other box for that would be like comparing apples with oranges.
That's one reason why I don't list Win2k or Win98 results, I have Win2k and 98 on other boxes with totally different hardware. This way everything is equal as far as hardware goes, because a benchmark comparison is total crap unless its done on a duel boot system. And im to lazy to set up duel boot NT,95,98,2K boxes :-) So XP is about it.... Tom
bye michael
hi, Michael Stefaniuc wrote on 07/28/2005 12:37 PM:
I do not think that MacOS X runs on his hardware. Using an other box for that would be like comparing apples with oranges.
hm, ok. what about native-linux ut2004 and q3, do they have benchmarks on linux, too? would be interesting to have the native version vs. wine version as well, wouldn't it? holger.
On 7/28/05, Holger Dell <devel(a)holger-dell.de> wrote:
hm, ok. what about native-linux ut2004 and q3, do they have benchmarks on linux, too? would be interesting to have the native version vs. wine version as well, wouldn't it?
Gentoo Demo 1 Demo2 388.3fps 391.0fps XP Demo 1 Demo2 355.7fps 353.3fps Wine Demo 1 Demo2 322.5fps 319.0fps ftp://ftp.idsoftware.com/idstuff/quake3/linux/ If you receive a error in check sums try this: export _POSIX2_VERSION=199209 1024x768x24 fullscreen = off ~ "timedemo 1" run the demo then ~ to see fps Anyone care to share there scores :-) Tom
holger.
From: "Tom Wickline" <twickline(a)gmail.com>
Is this what you guys had in mind? http://wiki.winehq.org/BenchMark
And just add a third column with +/- %'s
Any other Recommendations?
Why repeat the measurements? Something like this would be more desirable: Wine XP 3DMark Result: 14,606 marks 13,332 marks CPU Speed: 786 CPU Marks 650 CPU Marks ... Or even better, factor out the measurement unit too: Wine XP Wine vs. Win Unit 3DMark Result: 14,606 13,332 +10% marks CPU Speed: 786 650 +21% CPU Marks ... Or you can include the unit with the measure: Wine XP Wine vs. Win 3DMark Result (marks): 14,606 13,332 +10% CPU Speed (CPU Marks): 786 650 +21% ... I personally like the last one the best. -- Dimi Paun <dimi(a)lattica.com> Lattica, Inc.
On 7/28/05, Dimi Paun <dimi(a)lattica.com> wrote:
Wine XP Wine vs. Win 3DMark Result (marks): 14,606 13,332 +10% CPU Speed (CPU Marks): 786 650 +21% ...
I personally like the last one the best.
Okay #3 it is... Ill have time to do it the *correct* way on Saturday. What is there now will at least make it easy to view on bench against the other. Thanks, Tom
-- Dimi Paun <dimi(a)lattica.com> Lattica, Inc.
On Thu, 28 Jul 2005, Dimi Paun wrote: [...]
Wine XP Wine vs. Win 3DMark Result (marks): 14,606 13,332 +10% CPU Speed (CPU Marks): 786 650 +21% ...
I would exchange the Wine and XP columns. That would make the percentage column more understandable. -- Francois Gouget fgouget(a)free.fr http://fgouget.free.fr/ In theory, theory and practice are the same, but in practice they're different.
On 7/28/05, Dimi Paun <dimi(a)lattica.com> wrote:
Wine XP Wine vs. Win 3DMark Result (marks): 14,606 13,332 +10% CPU Speed (CPU Marks): 786 650 +21% ...
I personally like the last one the best.
I'm not finished, but I thought I would ask if this going in the right direction? http://wiki.winehq.org/BenchMark Tom
-- Dimi Paun <dimi(a)lattica.com> Lattica, Inc.
Tom Wickline wrote:
I'm not finished, but I thought I would ask if this going in the right direction? http://wiki.winehq.org/BenchMark
Much better, although Windows should really be on the left side. Felix
On 7/30/05, Felix Nawothnig <felix.nawothnig(a)t-online.de> wrote:
Tom Wickline wrote:
I'm not finished, but I thought I would ask if this going in the right direction? http://wiki.winehq.org/BenchMark
Much better, although Windows should really be on the left side.
Why? its Wine against XP ... not XP against Wine. But ill change it if others think it would be best :-) Tom
Felix
Tom Wickline wrote:
I'm not finished, but I thought I would ask if this going in the right direction? http://wiki.winehq.org/BenchMark Much better, although Windows should really be on the left side. Why? its Wine against XP ... not XP against Wine.
Maybe it's just me (no it's not, someone else in this thread also suggested it ;) but when I looked at it I really expected XP being on the left side. This is probably caused by the third column - a green +42% looks like the expected increase when switching from Windows to Wine for me. Another explanation would be that we are reimplementing the Windows API - first there was Windows, then Wine was written. I'd also like to note that this (Windows, Wine) is the same order as used on test.winehq.com: 95,98,ME => NT3,NT4,2K,XP,2K3 => Wine (consider that this chronology is in terms of popularity with the families grouped afterwards). It also implies that Wine will take over the world one day. :-) Felix
Tom Wickline wrote:
On 7/30/05, Felix Nawothnig <felix.nawothnig(a)t-online.de> wrote:
Tom Wickline wrote:
I'm not finished, but I thought I would ask if this going in the right direction? http://wiki.winehq.org/BenchMark
Much better, although Windows should really be on the left side.
Why? its Wine against XP ... not XP against Wine. But ill change it if others think it would be best :-)
Tom
Felix
.
Yes, in this case XP is the reference platform, so it should be on the left. Also, you could change the title of the comparison column to "Wine Difference" or some such. --Mitchell -- I find that a great part of the information I have was acquired by looking up something and finding something else on the way. -- Franklin P. Adams
On Sat, 2005-07-30 at 05:43 -0400, Tom Wickline wrote:
I'm not finished, but I thought I would ask if this going in the right direction? http://wiki.winehq.org/BenchMark
Looks great! I must admit that having XP on the left side may be preferable, if it's not too much work. -- Dimi Paun <dimi(a)lattica.com> Lattica, Inc.
On 7/30/05, Dimi Paun <dimi(a)lattica.com> wrote:
Looks great! I must admit that having XP on the left side may be preferable, if it's not too much work.
Does this look presentable? Any more recommendations? Anyone other than Mitchell think I should change "Wine vs. XP" to "Wine Difference" ? Tom
-- Dimi Paun <dimi(a)lattica.com> Lattica, Inc.
Tom Wickline wrote:
Does this look presentable?
Much better now.
Any more recommendations?
The % are apparently calculated wrong. You should use: (Wine score / XP score) - 1 For example in the first CPU Marks you say XP 650, Wine 786, +18% However, (786 / 650) - 1 = 0.2092307692307692 ~ +21% -- Anssi Hannula
Anssi Hannula wrote:
Tom Wickline wrote:
Any more recommendations?
The % are apparently calculated wrong.
You should use: (Wine score / XP score) - 1
For example in the first CPU Marks you say XP 650, Wine 786, +18%
However, (786 / 650) - 1 = 0.2092307692307692 ~ +21%
I don't know if you already read my post, but I see now some very strange numbers, like -326%. That would mean Wine had negative 3D marks :S Just use the above formula (e.g. Wine's score always first), no matter if wine has a better or worse result than XP. Wrong: 19197 / 5888 = 3.26 => -326% Right: (5888 / 19197) - 1 = -0.69 = -69% -- Anssi Hannula
On 7/30/05, Anssi Hannula <anssi.hannula(a)gmail.com> wrote:
I don't know if you already read my post, but I see now some very strange numbers, like -326%. That would mean Wine had negative 3D marks :S
:-)
Just use the above formula (e.g. Wine's score always first), no matter if wine has a better or worse result than XP.
Wrong: 19197 / 5888 = 3.26 => -326%
Right: (5888 / 19197) - 1 = -0.69 = -69%
Yea I know, I was goofing off and put the wrong ones in just to see what it looked like...... horrid to say the least... If you have a couple free minutes can you go over my %'s to make sure i didn't make a typo? BTW, Thanks! Tom
-- Anssi Hannula
The DronezMark scores from a couple days back are incorrect! I had VSync set in XP and that was the reason for Wine having such a huge % increase over XP. I re-ran the test on XP and Wine with sound set to Performance and VSync turned off. XP 1024x768x32 Wine 1024x768x24 Tom
On Wed, 2005-07-27 at 00:49, Tom Wickline wrote:
Hello,
I put the Benchmark results that I posted to wine-devel back in April on the Wiki. http://wiki.winehq.org/BenchMark
I would appreciate any constitutive criticism to improve this page.
Tom
Hi Tom, maybe it's an idea to put the WinXP and Wine results next to each other, to have some immediate idea of the differences. Maybe even add a diff column showing how well/bad we do against WinXP (or any other version tested). Paul.
On 7/27/05, Paul Vriens <Paul.Vriens(a)xs4all.nl> wrote:
Hi Tom,
Hello Paul,
maybe it's an idea to put the WinXP and Wine results next to each other, to have some immediate idea of the differences.
Okay, I'll try to do this by Saturday.
Maybe even add a diff column showing how well/bad we do against WinXP (or any > other version tested).
What I would like to do in the end is have two benchmark pages. Wine against XP and Wine old-version against wine current and show our progression/regression each six months or so. And have the diff column with +/- %'s for both pages. Thanks, Tom
Paul.
On Thu, 2005-07-28 at 00:35 -0400, Tom Wickline wrote:
What I would like to do in the end is have two benchmark pages. Wine against XP and Wine old-version against wine current and show our progression/regression each six months or so. And have the diff column with +/- %'s for both pages.
I think it would be best to have all these on one page, but different columns, so we can easily compare across. -- Dimi Paun <dimi(a)lattica.com> Lattica, Inc.
participants (14)
-
Anssi Hannula -
Anssi Hannula -
Christoph Frick -
Dimi Paun -
Dmitry Timoshkov -
Felix Nawothnig -
Francois Gouget -
Holger Dell -
Holger Dell -
Michael Stefaniuc -
Mitchell Mebane -
Oliver Stieber -
Paul Vriens -
Tom Wickline