Yesterday I added an option to let the TestBot administrator decide whether to run all the tests in Wine for every Wine patch. I have not activated it yet.
But I set it to run all of a dll's tests on Wine whenever that dll is patched. So for instance because job 43715 patched dinput the TestBot reran all of the dinput tests. In contrast, for dinput8 the patch only modified dinput8:dinput so only that test was rerun. And yes, this also illustrates that this mode does not take PARENTSRC into account (yet) :-(
https://testbot.winehq.org/JobDetails.pl?Key=43715&f601=win32.report#k60...
If this does not cause too many issues (false positives, long processing times) I'll change it to run all the tests all the time in a few days.
The remaining todo list for the TestBot:
* Add support for locales in Wine VMs. * Fix the performance issues caused by running all the tests on Wine for every Wine patch (I expect we can get by with just duplicating the Debian VM so we can run all the win32 tests on one host while another runs the module-level wow32 + wow64 + locale ones). * Allow submitting Windows test executables to run in Wine. * Allow specifying a locale for jobs running in Wine. * Better locale handling when submitting a job on the Windows VMs (duplicating the VM on the submit page is fine as long as there are only a couple locales but will get unwieldly as more are added). * Fix date/time handling on Wine VMs (the time is not updated when the task starts and risks getting too far off). * Support for Wine's other source repositories to limit the risk of misidentifying Wine patches. * Have the TestBot build its own WineTest executables. * Split the "Update Wine VMs" job in two so it does not hog the Debian VM for hours. * Testing vkd3d's patches... somehow. * Monitoring the VM disk sizes (the Debian one grows particularly fast). * Fix monitoring of WineTest log sizes. * Update the Windows VMs (Windows+TestAgentd). The Windows 10 one obviously but also the slow reverting ones (w7u 3mn, wvista 1mn, w864 1mn). * Automate creating locale snapshots for the Windows Ultimate VMs somehow (either via Ansible or through TestAgentd). * Add more locale snapshots for the Windows Ultimate VMs (w7u and wvistau64). * Remember the user's VM preferences for jobs. * Let users change their email, password, etc. * One day maybe: rewrite the web interface using "modern" technologies to avoid page reloads.
On Wed, 31 Oct 2018, Francois Gouget wrote: [...]
The remaining todo list for the TestBot:
- Add support for locales in Wine VMs.
This is done. Every patch is now tested in a sampling of locales: English of course, one non-English 'regular' locale (French), and two 'double-byte' locales (Japanese and Chinese). I expect to add right-to-left locales soon.
Note that some tests (gdi32:font) get a lot of errors in the Japanese and Chinese locales. Any extra fonts or other packages I should install?
- Allow submitting Windows test executables to run in Wine.
This is done. You can now upload your own test executables and pass them custom options to test Wine. Depending on their bitness the executables will be run in either the win32 or wow64 builds.
An improvement coming with the next point will be the ability to pick the Wine build to test (probably more useful for patches).
- Allow specifying a locale for jobs running in Wine.
This is coming soon.
-
Francois Gouget