Summary: * New w1064 VM with all Windows 10 fall releases. * Rebuilt w7u VM with dual-screen and some languages. * Cleaned up wvista VM. * The cw-rx460 Linux results are back. * The vm3 SSD died, will be replaced soon. * Test PCI-passthrough VM on the development TestBot.
w1064 -----
This is essentially the same as the w10pro64 VM but with test configurations for 1507, 1607, 1709, 1809, 1909 and the latest 2009.
The goal is to spread the testing load across two VM hosts. So w10pro64 will handle all (or most of) the languages, while w1064 will handle the older Windows 10 releases and a few extra configurations like dual-screen and test without elevated privileges (not yet present), etc.
I will try to update w10pro64 to the latest spring Windows releases while I will keep adding fall releases to w1064. So they should each get updated once a year.
w7u ---
I rebuilt this VM from scratch to get it on more recent QEMu virtual hardware; and only kept the latest configuration.
Here is the base configuration summary: [CPU:2*IvyBridge-IBRS RAM:2GB disk:scsi-unmap eth:e1000-metered snd:ich9 GPU:vga display:spice virtio:0.1.185 testagentd:1.8]
The most annoying part is getting Windows Update to work because the servers now require SHA2 but Windows 7 does not support it out of the box. So I had to find and manually install the right KB fix to get it going. Then Windows Update ran out of memory so I installed another KB fix so it would just take forever to install a measly ~160 updates.
So to summarize I installed SP1 manually, added KB4474419 to add SHA2 support, KB3050265 to prevent Windows Update from running out of memory, KB2852386 to get winsxs disk cleanup options, and finally installed all updates up to 2020-11-25.
Then I added a dual screen test configuration, a non-elevated privileges configuration and a few European languages.
I avoided languages with tricky scripts because I was not sure if Windows 7 would support them right (Vista does not). And I decided to pick a set that does not overlap with the ones the Windows 10 VM is already testing. Let me know if you need a different set of languages.
wvista ------
I flattened all the snapshots to only keep the latest version, that is all updates up to 2014-04-21.
Note that Vista runs into the same Windows Update issue as Windows 7: SHA2 support is required by the servers. But there's no way to get SHA2 support on Vista which means Vista has no Windows Update support at all.
I also tweaked the hardware configuration and updated the QEmu drivers. [CPU:1*SandyBridge-IBRS RAM:2GB disk:scsi-unmap eth:e1000 snd:ac97 GPU:vga display:spice virtio:0.1.185 testagentd:1.8]
The new configuration seems to be working pretty well except for test_driver4() in ntoskrnl.exe:ntoskrnl which crashes Vista. So I only run WineTest without elevated privileges which forces a skip of that test. But you can still run the tests with elevated privileges if you manually submit a job.
cw-rx460 Linux results ----------------------
cw-rx460 is not a TestBot VM but does run WineTest daily and has the advantage of running on real hardware. For a long time it failed to reliably submit WineTest results. Now it works again (the issue was in the wt-bot script).
vm3 SSD -------
The SSD of the vm3 host died last week so I moved its VMs to vm1 and vm4. That's part of why the TestBot was slow at the start of the week. The other reasons are:
* The Debian VM which is bloated and which I'm working on updating with a new libX11, * The couple older Debian configurations which I removed. * Too many VMs on vm1 (I moved some to vm4).
vm4's SSD is also on the brink of death according to smartctl so they'll both get new and bigger SSDs soon.
PCI-passthrough ---------------
I also set up a new Windows 10 Pro VM on my development TestBot environment and added a configuration with an RX550 graphics card to it using PCI-passthrough.
That seems to be working [1] so far, by which I mostly mean that besides actually running its TestBot tasks the graphics card has not locked up and required a host reboot yet.
What's annoying is that this VM crashes when it runs ntdll:exception so which prevents it from submitting its test results. That is however totally unrelated to the RX550 since it also happens with QEmu's built-in vga graphics card (a test configuration which has never seen the RX550 in any shape or form).
Another thing that's missing is support for screenshots but I figure that can come later (it would be great if someone wanted to tackle bug 44709).
There are many PCI-passthrough recipes online, but here's mine, just for adding the passthrough to an existing Windows VM (i.e. it assumes you've configured the host already):
* While still using QEmu's built-in vga card, install TightVNC (or equivalent).
I prefer to avoid Windows' RDP because that uses a separate desktop with its own graphics driver, changes the resolution, and disables sound. All that means I wouldn't really know what's going on with the actual graphics card.
* Then power off and virsh edit to: - replace the <domain...> line with <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> - add these lines before the closing </domain> (in fact it's likely fine anywhere) qemu:commandline <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.x-vga=on'/> </qemu:commandline>
* Remove the Display (Spice), Video (VGA), Tablet (USB|Virtio), and Sound (ich9) devices. Add the graphics card's two PCI devices (GPU + audio).
* Boot up, connect using VNC, install the graphics driver, reboot again.
* All done.
[1] One side effect is that whenever the RX550 configuration powers up it messes the colors of my desktop, as if it switched some palette. Fortunately all I have to do is to switch the focus to another window to fix it so I'm just putting up with it.
On Mon, 7 Dec 2020, Francois Gouget wrote: [...]
w1064
This is essentially the same as the w10pro64 VM but with test configurations for 1507, 1607, 1709, 1809, 1909 and the latest 2009.
It turns out that the 1909 snapshot was displaying the boot-time "Let's finish setting up your device" full-screen wizard (bug 50298). So I refreshed that snapshot to make sure it won't interfere with the tests.
I also added a configuration running the tests without elevated privileges (w1064_adm).
PCI-passthrough
I also set up a new Windows 10 Pro VM on my development TestBot environment and added a configuration with an RX550 graphics card to it using PCI-passthrough.
That seems to be working [1] so far, by which I mostly mean that besides actually running its TestBot tasks the graphics card has not locked up and required a host reboot yet.
And... now it's failing to initialize the RX550 driver with a code 43 error. Nothing seems to help so I suspect rebooting the host is going to be necessary. The next step is going to be checking whether performing clean shutdowns helps prevent this issue but that requires a TestBot patch.
debiant2 --------
This is a copy of debiant with the following changes: * I have updated the packages to the current Debian Testing. * I added fonts for the locales we can run the tests in. * I installed a manually backported X11 1.7.0 packages for bug 50086.
Unfortunately WineTest still times out.
user32:monitor is not the single culprit since WineTest sometimes times out before running it. I manually ran WineTest without a timeout and it took 33 minutes. There are 23 tests with runtimes above 30 seconds (only one timeout), representing close to 22 minutes!
Test Runtime (s) user32:monitor 120 cmd.exe:batch 93 dwrite:font 77 mlang:mlang 77 regedit.exe:regedit 67 ddraw:ddraw4 65 ddraw:ddraw7 65 d3d9:device 63 user32:msg 60 dxgi:dxgi 58 d3d11:d3d11 57 ddraw:ddraw2 56 dwrite:analyzer 54 ddraw:ddraw1 48 dwrite:layout 45 ws2_32:sock 42 d3d9:visual 41 d3d8:device 40 winhttp:winhttp 38 shell32:shlexec 36 d3d10core:d3d10core 35 d3d9:d3d9ex 35 reg.exe:reg 34
Maybe some of those can be sped up but I think it is going to be necessary is to increase the timeout to run the tests in Wine.
On Mon, 7 Dec 2020, Francois Gouget wrote: [...]
vm3 SSD
The SSD of the vm3 host died last week so I moved its VMs to vm1 and vm4. That's part of why the TestBot was slow at the start of the week. The other reasons are:
[...]
vm4's SSD is also on the brink of death according to smartctl so they'll both get new and bigger SSDs soon.
vm3 has a new SSD and is fully operational again. I moved the w1064 VM back to it which will give a bit of breathing room to vm1.
vm4 also has a new SSD. It is also back on the 4.19 kernel + QEmu 3.1 [1] which matches the configuration of the other VM hosts . To do so I had to change the pc-q35-5.0 machine to pc-q35-3.1 on w10pro64. As far as I can tell this makes no difference to the guest.
I will certainly move back to QEmu 5.0 one day but for now it did not seem to have any benefit. So sticking to QEmu 3.1 will avoid the "no Spice client" audio bug [2] and avoid any issue when moving VMs created on vm4 to the other hosts.
I also inadvertantly confirmed that the ntdll:exception crash is related to the kernel version: it happens with 4.19.0-8 but not with 4.19.0-13. Also this only impacts Windows 10: w7u never had any problem even with 4.19.0-8!
And while checking the WineTest results I noticed some getting this error:
C:\Users\Public\Documents>winetest64-latest.exe -q -s exe64.report The system cannot execute the specified program.
https://testbot.winehq.org/JobDetails.pl?Key=83433&f101=task.log#k101 https://testbot.winehq.org/JobDetails.pl?Key=83455&f101=task.log#k101
Yet Windows successfully ran WineTest just before. As far as I can tell this only happens on Windows 10 so I suspect Windows Defender since it does not seem possible to completely disable it on the recent versions. But this is not systematic so it's going to be hard to confirm.
It also happened last week so it's not related to the latest vm3 and vm4 update.
[1] Instead of kernel 5.8 + QEmu 5.0
[2] https://www.winehq.org/pipermail/wine-devel/2020-November/177510.html
So my understanding is that a snowstorm hit Minnesota and the power to the VM hosts went off. I expect them to come alive again when the power is restored which seems to be delayed by the holidays.
-
Francois Gouget