Prep for Wineconf - testing
Hi folks, Now that I have my room booked, my thoughts turn to what we'll actually do at Wineconf. And, of course, a Wineconf tradition is for me to lock everyone in the room, away from the bar, until the test situation improves. Sadly, despite the best efforts of some people (yay, Huw!), we are still staring at a range of test failures on Windows. It looks like we've got XP and 2003 starting to show green, and 2000 and 2008 look tantalizingly close. Looks like msi timeouts remain a problem. We had some discussion of throwing some hardware at the problem; I'm happy to do that, but, afair, it wasn't clear that it would help. Weren't we going to run an experiment by setting the run queue size to 1 to see if that prevented the msi timeout failures? If changing the run queue size from n to 1 and buying n machines will 'fix' this, maybe it's time to do that. I seem to recall that Francois needed cooperation from Alexandre to run that experiment - is that right? Once we think we've got robust Windows tests, then we move on to pursuit of the the holy grail - having at least two Linux machines capable of running make test cleanly. (I recall a glorious day when we achieved that; sadly, that moment didn't last :-/). So does it make sense to ask people to start daily winetest runs so we can populate a bit more data? Looks like we've got the CodeWeavers test boxes, and one from Austin. And I don't see any for Mac :-(. Cheers, Jeremy
I believe I answered these questions off-list but here are the answers so others know them too and can participate. On Mon, 27 Jul 2015, Jeremy White wrote: [...]
Weren't we going to run an experiment by setting the run queue size to 1 to see if that prevented the msi timeout failures? If changing the run queue size from n to 1 and buying n machines will 'fix' this, maybe it's time to do that.
To clarify the idea is to do nothing else while a test is running (and obviously only run one test at a time).
I seem to recall that Francois needed cooperation from Alexandre to run that experiment - is that right?
This is configured in testbot/lib/WineTestBot/Config.pm which Git currently indicates has the following settings: $MaxRevertingVMs = 1; $MaxRevertsWhileRunningVMs = 1; $MaxActiveVMs = 2; This indicates that we can have two active VMs, which means one running a test and the other idle, or two tests running simultaneously, or one test running while another VM is being reverted. But it's possible the settings on the winehq.org server are different. Only Alexandre has access to that machine. If so, then I guess the experiment would call for trying the following settings: $MaxRevertingVMs = 1; $MaxRevertsWhileRunningVMs = 0; $MaxActiveVMs = 1; I'm a bit skeptical that it would make much of a difference.
Once we think we've got robust Windows tests, then we move on to pursuit of the the holy grail - having at least two Linux machines capable of running make test cleanly.
Patches accepted.
And I don't see any for Mac :-(.
I should buy a Mac but I don't really have the time... Too bad we don't have any Wine Mac developers! -- Francois Gouget <fgouget(a)free.fr> http://fgouget.free.fr/ La terre est une bêta...
I hack on Wine for Mac. Anything special someone wants? -Matt
On Aug 31, 2015, at 7:18 PM, Francois Gouget <fgouget(a)free.fr> wrote:
I believe I answered these questions off-list but here are the answers so others know them too and can participate.
On Mon, 27 Jul 2015, Jeremy White wrote: [...]
Weren't we going to run an experiment by setting the run queue size to 1 to see if that prevented the msi timeout failures? If changing the run queue size from n to 1 and buying n machines will 'fix' this, maybe it's time to do that.
To clarify the idea is to do nothing else while a test is running (and obviously only run one test at a time).
I seem to recall that Francois needed cooperation from Alexandre to run that experiment - is that right?
This is configured in testbot/lib/WineTestBot/Config.pm which Git currently indicates has the following settings:
$MaxRevertingVMs = 1; $MaxRevertsWhileRunningVMs = 1; $MaxActiveVMs = 2;
This indicates that we can have two active VMs, which means one running a test and the other idle, or two tests running simultaneously, or one test running while another VM is being reverted.
But it's possible the settings on the winehq.org server are different. Only Alexandre has access to that machine. If so, then I guess the experiment would call for trying the following settings:
$MaxRevertingVMs = 1; $MaxRevertsWhileRunningVMs = 0; $MaxActiveVMs = 1;
I'm a bit skeptical that it would make much of a difference.
Once we think we've got robust Windows tests, then we move on to pursuit of the the holy grail - having at least two Linux machines capable of running make test cleanly.
Patches accepted.
And I don't see any for Mac :-(.
I should buy a Mac but I don't really have the time... Too bad we don't have any Wine Mac developers!
-- Francois Gouget <fgouget(a)free.fr> http://fgouget.free.fr/ La terre est une bêta...
On Mon, 31 Aug 2015, Matt Durgavich wrote:
I hack on Wine for Mac. Anything special someone wants?
I was the only one to run WineTest on Mac and since I no longer have a Mac the Mac column on the test.winehq.org website remains desperately empty (gone even). So we'd need someone to run WineTest daily on a Mac so we can track what's broken on the Mac compared to Linux. You get bonus points if you run WineTest twice: once with the Mac/Quartz driver and once with the X11 one so we can identify which issues are specific to the Mac driver. -- Francois Gouget <fgouget(a)free.fr> http://fgouget.free.fr/ Indifference will certainly be the downfall of mankind, but who cares?
Francois Gouget <fgouget(a)free.fr> writes:
But it's possible the settings on the winehq.org server are different. Only Alexandre has access to that machine. If so, then I guess the experiment would call for trying the following settings:
$MaxRevertingVMs = 1; $MaxRevertsWhileRunningVMs = 0; $MaxActiveVMs = 1;
I'm a bit skeptical that it would make much of a difference.
OK I changed it, let me know how it goes. -- Alexandre Julliard julliard(a)winehq.org
On 1 Sep 2015, at 06:14, Alexandre Julliard wrote:
Francois Gouget <fgouget(a)free.fr> writes:
But it's possible the settings on the winehq.org server are different. Only Alexandre has access to that machine. If so, then I guess the experiment would call for trying the following settings:
$MaxRevertingVMs = 1; $MaxRevertsWhileRunningVMs = 0; $MaxActiveVMs = 1;
I'm a bit skeptical that it would make much of a difference.
OK I changed it, let me know how it goes.
Early indications are that this does indeed help See: http://test.winehq.org/data/tests/msi:action.html (ignore the XP failures, they're not testbot's) and: http://test.winehq.org/data/tests/gdi32:font.html I don't think it's entirely surprising given the long revert times we're seeing. It seems to me that adding more hardware (to have fewer VMs per host) is the pragmatic way forward. Sure, we can argue that the long revert times are crazy and should be fixed, but nobody is going to do this anytime soon. Of course somebody has to setup the said new hardware and reconfigure testbot to use it, but hopefully that's something we know how to do. Huw.
It seems to me that adding more hardware (to have fewer VMs per host) is the pragmatic way forward. Sure, we can argue that the long revert times are crazy and should be fixed, but nobody is going to do this anytime soon. Of course somebody has to setup the said new hardware and reconfigure testbot to use it, but hopefully that's something we know how to do.
Hardware is cheap, particularly if we don't need beefy machines, as the expected load is lower. How many boxes should we buy? If I read Francois's notes on the default configuration, it seems like the pipe was originally '3', which implies two more boxes should do it. That sound right? And would a pair of i5s with fast hard drives and 8G of RAM do the trick? Cheers, Jeremy
On Thu, 3 Sep 2015, Jeremy White wrote: [...]
How many boxes should we buy? If I read Francois's notes on the default configuration, it seems like the pipe was originally '3', which implies two more boxes should do it.
Yes, two extra boxes should maintain the old status-quo.
That sound right? And would a pair of i5s with fast hard drives and 8G of RAM do the trick?
One aspect that I think is important is to ensure that the processor support the virtualization instructions. For Intel that's VT-x and VT-d and Intel tends to turn these features on and off at random across their processor range. So before buying a processor I'd recommend checking it out on http://ark.intel.com/. -- Francois Gouget <fgouget(a)free.fr> http://fgouget.free.fr/ "Lotto: A tax on people who are bad at math." -- unknown "Windows: Microsoft's tax on computer illiterates." -- WE7U
Yes, two extra boxes should maintain the old status-quo.
That sound right? And would a pair of i5s with fast hard drives and 8G of RAM do the trick?
One aspect that I think is important is to ensure that the processor support the virtualization instructions. For Intel that's VT-x and VT-d and Intel tends to turn these features on and off at random across their processor range. So before buying a processor I'd recommend checking it out on http://ark.intel.com/.
And no special graphics hardware, right, so Intel built in video is sufficient? If so, Newman, can you order up a pair of boxes (and a new UPS) that meet the criteria? Ideally, if we can get them online a little before Wineconf, we can work based on those results. Cheers, Jeremy p.s. And then anyone want to volunteer to help Francois? That sounds like it would be even more valuable...
Well, I have an old 2009 MBP with a Nvidia GPU I don't use anymore. It uses the old core 2 duo chip so not sure if it might be as useful as the new ones with i5's out there.. On Thu, Sep 3, 2015 at 8:28 PM, Jeremy White <jwhite(a)codeweavers.com> wrote:
Yes, two extra boxes should maintain the old status-quo.
That sound right? And would a pair of i5s with fast hard drives and 8G of RAM do the trick?
One aspect that I think is important is to ensure that the processor support the virtualization instructions. For Intel that's VT-x and VT-d and Intel tends to turn these features on and off at random across their processor range. So before buying a processor I'd recommend checking it out on http://ark.intel.com/.
And no special graphics hardware, right, so Intel built in video is sufficient?
If so, Newman, can you order up a pair of boxes (and a new UPS) that meet the criteria? Ideally, if we can get them online a little before Wineconf, we can work based on those results.
Cheers,
Jeremy
p.s. And then anyone want to volunteer to help Francois? That sounds like it would be even more valuable...
On Thu, Sep 3, 2015 at 9:55 PM, Jeremy White <jwhite(a)codeweavers.com> wrote:
Hardware is cheap, particularly if we don't need beefy machines, as the expected load is lower.
...
That sound right? And would a pair of i5s with fast hard drives and 8G of RAM do the trick?
Since sending email is free and the worst that can happen is getting a no as answer I would like to give an idea. Attaching a 15 USD smart card reader to any of them and making it work in Windows would make winscard support get alive in Wine. Inserting any old credit card with chip (very common at least in Brazil) or any other 1 USD contact card would make all kinds of tests work. A sample reader that also should be supported by pcsclite is at [1] [1] http://www.amazon.com/HID-Global-Corporation-R30210015-1-OMNIKEY/dp/B003BKV4... Best wishes, Bruno
On 09/03/2015 09:55 AM, Bruno Jesus wrote:
On Thu, Sep 3, 2015 at 9:55 PM, Jeremy White <jwhite(a)codeweavers.com> wrote:
Hardware is cheap, particularly if we don't need beefy machines, as the expected load is lower.
...
That sound right? And would a pair of i5s with fast hard drives and 8G of RAM do the trick?
Since sending email is free and the worst that can happen is getting a no as answer I would like to give an idea.
Attaching a 15 USD smart card reader to any of them and making it work in Windows would make winscard support get alive in Wine. Inserting any old credit card with chip (very common at least in Brazil) or any other 1 USD contact card would make all kinds of tests work. A sample reader that also should be supported by pcsclite is at [1]
[1] http://www.amazon.com/HID-Global-Corporation-R30210015-1-OMNIKEY/dp/B003BKV4...
Just to clarify - it wouldn't be attaching just one reader, right? If we have 4 test systems, then ideally, we'd attach 4 of these readers, so that all the machines would test identically, right? And we would need to add a kvm option to attach this device when we started kvm, but that shouldn't be hard. Cheers, Jeremy
On Fri, Sep 4, 2015 at 11:58 PM, Jeremy White <jwhite(a)codeweavers.com> wrote:
Attaching a 15 USD smart card reader to any of them and making it work in Windows would make winscard support get alive in Wine. Inserting any old credit card with chip (very common at least in Brazil) or any other 1 USD contact card would make all kinds of tests work. A sample reader that also should be supported by pcsclite is at [1]
[1] http://www.amazon.com/HID-Global-Corporation-R30210015-1-OMNIKEY/dp/B003BKV4...
Just to clarify - it wouldn't be attaching just one reader, right? If we have 4 test systems, then ideally, we'd attach 4 of these readers, so that all the machines would test identically, right?
Thanks for the reply, I was thinking about an overly simple solution where we could have a single reader connected to a physical machine and somehow exposed to a single virtual machine which would run XP or Windows 7. My virtualization knowledge is limited to vmware player so all I know is to right click and attach the usb reader, sorry =) The differences between XP and after XP for the main winscard functions are minimal so I don't think we need more than one reader to do it but certainly it wouldn't hurt. As for the reader brand/model the only important thing is to ensure the reader is PC/SC compatible, because some may be proprietary driver driven or simplified card ID to HID converters. Best wishes, Bruno
As for the reader brand/model the only important thing is to ensure the reader is PC/SC compatible, because some may be proprietary driver driven or simplified card ID to HID converters.
I had Jer order 4 SCR 3310 card readers. I know first hand that they work with pcsclite. Cheers, Jeremy
On 09/11/2015 10:11 AM, Jeremy White wrote:
As for the reader brand/model the only important thing is to ensure the reader is PC/SC compatible, because some may be proprietary driver driven or simplified card ID to HID converters.
I had Jer order 4 SCR 3310 card readers. I know first hand that they work with pcsclite.
And they are now attached to the vms. Bruno, holler if you need something configured for that to work. Right now, I don't have any smartcards inserted. Maybe let's start simple, and add them later. (I have a few smartcards, but not 4 identical ones :-/). Cheers, Jeremy
On Thu, Sep 17, 2015 at 2:15 AM, Jeremy White <jwhite(a)codeweavers.com> wrote:
On 09/11/2015 10:11 AM, Jeremy White wrote:
As for the reader brand/model the only important thing is to ensure the reader is PC/SC compatible, because some may be proprietary driver driven or simplified card ID to HID converters.
I had Jer order 4 SCR 3310 card readers. I know first hand that they work with pcsclite.
And they are now attached to the vms. Bruno, holler if you need something configured for that to work. Right now, I don't have any smartcards inserted. Maybe let's start simple, and add them later. (I have a few smartcards, but not 4 identical ones :-/).
Thanks for the effort. It's ok to have different cards, each card may follow different specs so we are not interested in what they do or not, just that they work in the readers so SCardConnect and SCardTransmit works.
On Thu, 3 Sep 2015, Huw Davies wrote: [...]
Early indications are that this does indeed help See: http://test.winehq.org/data/tests/msi:action.html (ignore the XP failures, they're not testbot's) [...] and: http://test.winehq.org/data/tests/gdi32:font.html [...] I don't think it's entirely surprising given the long revert times we're seeing.
On the contrary I find that surprising. My understanding is that the MSI tests are I/O bound. But given the long revert times, reverts are not doing much I/O at all so they should not interfere that much with the tests. However based on some times I looked at this recently the QEmu process spins its heels, seemingly doign nothing. That means it keeps a core fully occupied. But given that the VM hosts have at least 4 cores each that should still not slow down the MSI tests much given that they only have access to two cores anyway.
It seems to me that adding more hardware (to have fewer VMs per host) is the pragmatic way forward.
It would also help to have more people contributing to the TestBot. -- Francois Gouget <fgouget(a)free.fr> http://fgouget.free.fr/ I haven't lost my mind, it's backed up on tape around here somewhere...
participants (7)
-
Aaryaman Vasishta -
Alexandre Julliard -
Bruno Jesus -
Francois Gouget -
Huw Davies -
Jeremy White -
Matt Durgavich