Wine test week 2 - nice apparent progress
My script results are here: http://www.winehq.org/~jwhite/ebd5f96aea22.html We had a lot of tests that had failed on Windows 7 ultimate that we now skip; that has reduced the number of apparent failures on the new testbot machines down under 20. We've also had some nice work from folks like Dmitry (thanks!). I'll keep annotating the failures until they all have notes. I do have one that could use a sucker^H^H^H^H^H^Hvolunteer: http://test.winehq.org/data/ebd5f96aea22f4093d4ceb109bdead2657666c57/xp_newt... I had asked Misha Koshelev (the apparent last person to meaningfully change those tests) if he wanted to look into it, but he's in the middle of a medical internship working brutal hours, so he demurred. Cheers, Jeremy
On 25 March 2014 01:00, Jeremy White <jwhite(a)codeweavers.com> wrote:
My script results are here: http://www.winehq.org/~jwhite/ebd5f96aea22.html
For what it's worth, note that the intermittent failures in ddraw:ddraw7 are timeouts, and happening in different places. I suspect that's either the generic issue of the testbot timing out on occasion, or the testbot just not being fast enough to finish the test in time, perhaps due to it being under some load from a different VM.
On 03/25/2014 02:33 AM, Henri Verbeet wrote:
On 25 March 2014 01:00, Jeremy White <jwhite(a)codeweavers.com> wrote:
My script results are here: http://www.winehq.org/~jwhite/ebd5f96aea22.html
For what it's worth, note that the intermittent failures in ddraw:ddraw7 are timeouts, and happening in different places. I suspect that's either the generic issue of the testbot timing out on occasion, or the testbot just not being fast enough to finish the test in time, perhaps due to it being under some load from a different VM.
Francois, do we have (or could we easily create) a mechanism to test that by constraining the number of VMs that run simultaneously? I know it would slow performance, but it would be interesting to see if we can quantify that. Hardware is cheap; we can easily throw more testbots at the problem... Cheers, Jeremy
On 03/25/2014 01:51 PM, Jeremy White wrote:
On 03/25/2014 02:33 AM, Henri Verbeet wrote:
On 25 March 2014 01:00, Jeremy White <jwhite(a)codeweavers.com> wrote:
My script results are here: http://www.winehq.org/~jwhite/ebd5f96aea22.html
For what it's worth, note that the intermittent failures in ddraw:ddraw7 are timeouts, and happening in different places. I suspect that's either the generic issue of the testbot timing out on occasion, or the testbot just not being fast enough to finish the test in time, perhaps due to it being under some load from a different VM.
Francois, do we have (or could we easily create) a mechanism to test that by constraining the number of VMs that run simultaneously? I know it would slow performance, but it would be interesting to see if we can quantify that. Just two run in parallel. I wanted to ask Francois to bump that number up not lower it. The testbot is basically unusable for hours during the daily test run due to that.
Hardware is cheap; we can easily throw more testbots at the problem... I got told repeatedly by well informed sources that that isn't needed. Just sayin'
bye michael
Francois, do we have (or could we easily create) a mechanism to test that by constraining the number of VMs that run simultaneously? I know it would slow performance, but it would be interesting to see if we can quantify that. Just two run in parallel. I wanted to ask Francois to bump that number up not lower it. The testbot is basically unusable for hours during the daily test run due to that.
Ah. Okay. Yeah, I think there is a sense that we want the software to behave sensibly before we start throwing hardware at the problem.
Hardware is cheap; we can easily throw more testbots at the problem... I got told repeatedly by well informed sources that that isn't needed. Just sayin'
Actually, we've got a new server specced out, and we're going to put it into production use next week. If that goes well, we'll likely plan a migration for WineHQ as well... Cheers, Jeremy
Am 25.03.2014 16:54, schrieb Jeremy White:
Hardware is cheap; we can easily throw more testbots at the problem... I got told repeatedly by well informed sources that that isn't needed. Just sayin'
Actually, we've got a new server specced out, and we're going to put it into production use next week. If that goes well, we'll likely plan a migration for WineHQ as well...
what about also migrating the wiki to your site? (including the code to Wines git server)
On 2014-03-25 07:51-0500 Jeremy White wrote:
On 03/25/2014 02:33 AM, Henri Verbeet wrote:
On 25 March 2014 01:00, Jeremy White <jwhite(a)codeweavers.com> wrote:
My script results are here: http://www.winehq.org/~jwhite/ebd5f96aea22.html
For what it's worth, note that the intermittent failures in ddraw:ddraw7 are timeouts, and happening in different places. I suspect that's either the generic issue of the testbot timing out on occasion, or the testbot just not being fast enough to finish the test in time, perhaps due to it being under some load from a different VM.
Francois, do we have (or could we easily create) a mechanism to test that by constraining the number of VMs that run simultaneously? I know it would slow performance, but it would be interesting to see if we can quantify that.
I am not a wine developer, but observing this interesting thread from the outside point of view, it seems to me what you really need is a mechanism to specify "timeout" for each test not in terms of wall-clock time, but in terms of roughly how many floating-point operations are consumed by the test before it is decided the test is a failure. IOW, each timeout in seconds should effectively be multiplied by the integral of the time-variable load factor on the actual hardware that is running the VM divided by a constant factor corresponding to the speed of the machine. So for example, the timeout in seconds could be specified for a 1-GHz otherwise idle theoretical hardware, and adjusted appropriately on-the-fly to speed of the actual hardware and its time-variable load. If this is implemented correctly, once a test succeeds with a given timeout (specified for the theoretical hardware) on given hardware, it should never fail again due to load/computer speed factors on that or any other hardware with that same given timeout. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________
On Tue, 25 Mar 2014, Jeremy White wrote: [...]
Francois, do we have (or could we easily create) a mechanism to test that by constraining the number of VMs that run simultaneously?
Currently the way to do so is to send a patch to modify $MaxActiveVMs, and/or $MaxRevertsWhileRunningVMs in testbot/lib/WineTestBot/Config.pm to Alexandre. However the old VM host had an 8 core CPU and the limits are to run at most 2 VMs at once, and each VM has only 2 cores (except the build VM which had 4). So really there should not have been much contention on the cores (which it seems like should be all that matters for these tests). The new VM host has only 4 cores (+hyperthreading). We're still only running two VMs at a time but now the build VM only has 2 cores. We can do a test with $MaxActiveVMs = 1 and $MaxRevertsWhileRunningVMs = 0 but that would really limit performance. -- Francois Gouget <fgouget(a)codeweavers.com>
participants (6)
-
Alan W. Irwin -
André Hentschel -
Francois Gouget -
Henri Verbeet -
Jeremy White -
Michael Stefaniuc