Dan Kegel wrote:
(This was last discussed in February, e.g. http://www.winehq.org/pipermail/wine-devel/2009-February/073060.html )
The results on test.winehq.org seem more variable than one would expect, which makes it harder to gauge wine's progress.
I can think of two sources of noise:
- 32 and 64 bit results are mixed together
- we don't have a stable stable (sic) of machines
running the tests
Removing these two sources of noise might be as simple as
- omit 64 bit results, and
- omit computers for which results are not
consistently available throughout the time range being displayed
Shouldn't be too hard for someone to whip together an alternate report that did that. Wish I had the time to...
- Dan
So what constitutes a stable machine? I for one am using VMware for most of my boxes which are clean installs but up-to-date with patches and such. After running winetest they will be reverted to the last correct snapshot (winetest still leaves a lot of rubbish around that could potentially influence a next run).
Side note: One thing I'd also like to see (on first glance) on test.winehq.org is whether we are dealing with a real box or a virtualized one.
Side note 2: Our index per build pages are almost 4MB in size. Splitting things up will also cut down on that.