[Bug 39425] New: Improve resilience to VM host outages

https://bugs.winehq.org/show_bug.cgi?id=39425

Bug ID: 39425
           Summary: Improve resilience to VM host outages
           Product: Wine-Testbot
           Version: unspecified
          Hardware: x86
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

Currently the WineTestBot gets stuck when the connection to the libvirt server
on the VM hosts is broken. This includes cases where the libvirt server is
restarted, the VM hosts is rebooted or cases where there's a network outage.

The reason is that the Engine queries the status of the VMs itself in some
circumstances. This creates a TCP connection which is never recreated in case
it breaks.

The proper fix is to banish all such queries from the Engine: not just to fix
this issue but also because some of these operation can be long (a few seconds)
and block the main loop of the single-threaded Engine, which can in turn cause
the website to lag.