w11pro64_nv went offline a couple of times. That's because QEmu was unable to use the graphics card, issuing an "Unknown PCI header type ‘127’" error. This typically happens when the GPU is left in a bad (~sleep) state after the VM using it is forcefully powered off. The only fix is to reboot the host. Yuck!
I think this happened because Windows Update was not really disabled (see below) and took its sweet time updating the VM on shutdown, causing a timeout and thus a forceful power off.
So I updated the Windows 11 VM and hopefully this will not happen anymore.
So what's new with the Windows 11 VM? -------------------------------------
* It now has all the 21H2 updates up to 2022-11-04. (see also 'What about Windows 11 22H2?' below)
* On Windows 10 setting a data cap would automatically also mark the network connection as metered. However in Windows 11 this is no longer the case and it is the metered part that prevents Windows from updating. So when I went through my standard procedure to set up this VM, Windows update was not disabled at all. So in this update I fixed this and now neither Edge nor Windows should be updating while we are running the tests.
* I also changed the Edge and Teams default configuration so they don't start at boot.
* And I cleared a couple of tasks in the security center and set the password to not expire monthly! (Is that local account default another way for Microsoft to push users to create an online account?)
* I changed the w11pro64_amd and w11pro64_nv test configurations to only install the graphics driver and not the corresponding fancy GUIs. This should prevent the QT component of AMD's GUI from interferring with the clipboard tests.
w11pro64_amd has whql-amd-software-adrenalin-edition-22.5.1-win10-win11-may10.exe w11pro64_nv has nvidia-517.48-desktop-win10-win11-64bit-international-dch-whql.exe
* After the update I could not get the AMD RX 6600 graphics card to work with PCI passthrough anymore: the Windows driver would just show a (generic) 'code 43' error.
Fortunately at some point I found vfio-pci errors on the host in kernel.log: vfio-pci 0000:03:00.0: BAR 0: can't reserve [mem 0x4020000000-0x402fffffff 64bit pref]
Then /proc/iomem showed that part of that memory range was used by the efifb module which some resources online recommend disabling for PCI passthrough. And indeed that fixed things!
It's strange that this module did not cause trouble before. I suspect it is quite recent and only got loaded when I rebooted the host to fix the NVIDIA graphics card!
As usual, see the WineTestBot VMs wiki page for details of how the TestBot VMs are set up:
https://wiki.winehq.org/Wine_TestBot_VMs#PCI-Passthrough
Audio vs. PCI passthrough -------------------------
The regular VMs have an ich9 emulated audio card. That sometimes causes trouble when tests do short duration timing tests because of the way Spice's reports its buffering to keep the audio and video in sync over the network.
So the PCI passthrough VMs do away with ich9 to the tests to use the graphics cards sound support instead, that is audio through HDMI / DisplayPort. That's real hardware so it should be perfect!
But there's a hitch: the 'screen' that these graphics cards are connected to don't support audio. The graphics card detect that and report that no 'speakers' are connected. One of the visible impact is that this causes Windows to set the sound level to zero and prevent the user from changing the volume. And it causes some Wine test units (see mmdevapi:*) to skip the tests [1]. They are probably right to but it's just galling to have real hardware and not be able to fully use it :-(
I put 'screen' in quotes above because in fact the TestBot VM hosts are rack mounted and use 'dummy HDMI / DisplayPort plugs'. The descriptions on online stores never specify whether this type of plug supports audio which I take to mean that probably none of them does (if you know of any that do let me know).
I guess the next step will be to test this with the KVM instead but it's not clear if it supports audio through HDMI (online resources are so useless for these products!). At least the HDMI KVM dongles advertise audio support but don't have a jack connector and lsusb does not show any audio device. So there it looks like audio would have to go through HDMI.
[1] mmdevapi:mmdevenum has failures probably because it does not detect this case.
What about Windows 11 22H2? ---------------------------
First Windows 11 will not automatically install it through Windows Update because that involves a recheck that the PC is compatible which obviously fails since the VM has neither UEFI nor TPM 2.0 (see 'QEmu vs. UEFI' below).
But booting on the 22H2 ISO (or Rufus' USB 'key') results in an immediate freeze. It turns out that can be avoided by changing the emulated VM 'chipset' from Q35 to i440FX.
That's a bummer because for PCI passthrough Q35 is much recommended because it emulates PCI-Express which matches the bus the actual graphics cards and their drivers expect. So essentially the choice is between PCI passthrough and Windows 11 22H2.
However online there are few reports of trouble with 22H2 and Q35. So I suspect this is a QEmu 5.2 bug and that upgrading would likely fix that. So I tabled the 22H2 upgrade and will revisit it when I can update QEmu. The VM hosts run Debian Stable which only has QEmu 5.2. Debian Testing has version 7.1 so I'll see if that can easily be brought there (maybe via backports).
QEmu vs. UEFI -------------
As explained on the WineTestBot VMs page, the normal way to set up a Windows 11 VM in QEmu would be to configure the VM to use UEFI to get support for secure boot and to add an emulated TPM.
Similarly a number of online resources recommend configuring the VM with UEFI for PCI passthrough, though those seem to be from the early PCI passthrough days and no longer seem to be relevant.
However there is a blocker that prevents using this approach for the TestBot: Libvirt does not support taking snapshots of UEFI VMs, whether live or not. More precisely internal qcow2 snapshots, the only type supported by Libvirt, are incompatible with UEFI. The QEmu developers won't fix that and instead recommend using external snapshots. But that requires a lot of work to be supported in Libvirt (i.e. it could still take years). Currently it would be possible to create an external disk-only snapshot by changing the TestBot's Libvirt code but it would not be possible to restore it without running QEmu manually (i.e. give up on the snapshot VM XML description file and Libvirt's remote access support). So that rules out UEFI entirely.
So this Windows 11 VM uses a regular BIOS and relies on Rufus to strip Windows 11's UEFI, secure boot and TPM requirements.
https://wiki.winehq.org/Wine_TestBot_VMs#Windows_11