[Bug 31786] New: Run the tests on real hardware

http://bugs.winehq.org/show_bug.cgi?id=31786

Bug #: 31786
           Summary: Run the tests on real hardware
           Product: Wine-Testbot
           Version: unspecified
          Platform: x86
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: unknown
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]
    Classification: Unclassified

It would be very useful to run some tests on real hardware.
* This is the case for all the 3D graphics tests. For these we would even want
to run them on both AMD and NVidia graphics cards(*). Currently they are
essentially ignored by WineTestBot because the virtual graphic cards don't
provide meaningful 3D support.
* It could somewhat be useful for sound tests too, mostly to avoid timing
issues, but also to ensure the results we get are not tainted by bugs in the
emulated sound card. Also emulated sound cards typically don't support MIDI
emulation.

The problem is that it's very important to limit the test duration and to
restore the test environment to a clean state after each test, no matter what
happens, for two reasons:
* Because it's not uncommon for tests to have side effects and for them to
leave the test environment in an unusable state. Granted it happens much more
often with older Windows releases as they are more prone to crashes. However
graphics drivers also tend to be buggy and we would not want the tests to be
tainted by a previous run (though a reboot should take care of most issues).
* But also the Wine TestBot runs anything that gets posted by anonymous users
on wine-patches. That's ok as long as such code is properly confined and not
allowed to run for too long. That supposes thoroughly cleaning up the
environment after a test has run, including anything that could happen at boot
time.

Virtual machines provide a simple solution to both of these issues which is why
the TestBot relies on them heavily.

With QEmu it's possible to let the guest OS access some peripherals directly
via the 'passthrough' mode. This is normally done for network cards. If it
worked for graphics cards that would be a great solution but it's not clear
that it's feasible.

Another option would be to play tricks with Grub scripts: have it boot from a
clean Linux partition (or from the network?), restore the Windows partition
from a clean disk image, reboot into that partition, and back on the clean
partition with the next reboot. That still leaves the issue of forcing a reboot
in case of a freeze or malware taking control of Windows (use a remote
controlled power switch like in data centers?).

(*) We currently have two such machines ready for this:
http://www.winehq.org/pipermail/wine-devel/2012-February/094080.html