New subject: [Bug 47855] Run tests in more than one VM at a time

30 Sep 2019


      https://bugs.winehq.org/show_bug.cgi?id=47855
Bug ID: 47855
           Summary: Run tests in more than one VM at a time
           Product: Wine-Testbot
           Version: unspecified
          Hardware: x86
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: unknown
          Assignee: wine-bugs@winehq.org
          Reporter: fgouget@codeweavers.com
      Distribution: ---
Some Wine tests have pretty stringent timing requirements:
* Some audio tests play a short sound bite and make sure the playback lasted
for the expected time within a few tens of seconds as a way to ensure audio is
played at the right rate. But VMs are not always very good at respecting
precise timing requirements when there is other activity on the host.
* Some tests (msi) generate quite a bit of disk traffic and take a while to
run. Running two of those simultaneously can sometimes push them over the 2
minutes limit, causing them to time out. That said there may be something else
afoot here as these run times vary from ~20s to ~100s depending on the VM for
seemingly no reason.
As a consequence the TestBot is configured to only do one thing at a time: run
at most one test or revert one VM. This is at odds with hardware evolution and
new TestBot requirements:
* SSDs are much faster than hard drives and thus should solve the disk
bottleneck issues.
* The builds for the Wine VMs take a lot more time than just producing PE
executables for the Windows VMs. To speed up compilation it is necessary to
increase the parallelism which means buying a machine with as many cores as
possible and assigning as many as possible to the Wine VMs.
* But while running single threaded tests in a single VM on a quad-core machine
is bad enough, doing the same thing on a 16+ threads machine would truly be a
waste.
So a way to be able to run tests in more than one VM at a time should be found.
The first step would be to verify that these issues still exist (this means
distinguishing them from the usual random failures). If they don't then problem
solved!
Otherwise CPU-pinning is a solution that's usually mentioned in gaming circles.
The questions are:
* Would it actually help?
* How to integrate it with the TestBot. Is this something that should be
configured through Libvirt when restoring a live snapshot? Or is this something
that should be defined 'statically' in the VM's configuration?
* What happens if two VMs are pinned to the same CPU core? Will Libvirt prevent
us from starting both at the same time or is it up to the TestBot Engine to
make sure this does not happen? How?
* What happens when a VM that's is pinned to CPU core 7 is moved to a 4 core
host? Is the pinning ignored or does the administrator have to manually change
its configuration?
-- 
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.

[Bug 47855] New: Run tests in more than one VM at a time