TL;DR; - Part 2 of the todo list, focused on the web server side and TestBot reliability. - False positives caused by intermittent failures should be much rarer. - Two other false positive sources are close to being fixed.
* Avoid false positives when intermittent failures happen. https://bugs.winehq.org/show_bug.cgi?id=47998 -> Done.
* Provide an efficient way to detect new failures on test.winehq.org https://bugs.winehq.org/show_bug.cgi?id=48164 - This is needed to properly monitor the WineTest results and detect regressions so this is a pretty high priority issue. -> Now that the intermittent failures are out of the way I plan to fix a few small bugs that cause false positives and then work on this. (cue jokes about plans and battlefields...)
* Fix handling of child processes. - winetest_wait_child_process() is wrong in many ways. - But the TestBot also needs changes to detect when a child process crashed and avoid generating a false positive. -> Wrote WTBS test cases. The Wine and TestBot patches are in progress.
* Better handle patch series: - Don't mix up different versions of the same patch series https://bugs.winehq.org/show_bug.cgi?id=48353 -> Patches written, need more testing, queued behind other patches. - Don't overwrite existing patch series parts https://bugs.winehq.org/show_bug.cgi?id=47042 -> Patch written and tested.
* Automatically update the WinePrefix if a patch requires it - The TestBot prepares the WinePrefixes in advance so it does not waste time creating them on the fly for each patch. Unfortunately some patches require updating the WinePrefix and this causes them to fail. https://bugs.winehq.org/show_bug.cgi?id=48354 - The first step is to make use of shared Gecko and Mono installs to speed up the WinePrefix creation. Then it may become acceptable to create WinePrefixes on the fly. https://bugs.winehq.org/show_bug.cgi?id=48652 -> I have updated wt-daily to support shared Gecko and Mono installs (so I know how to do it). Now this just needs to be replicated in the TestBot.
* Limit the size of the task reports and logs - A recent infinite loop bug in kernel32:process caused it to generate 140+ MB of test failures. The TestBot then blindly downloaded the reports and parsed them. But besides the disk usage on the TestBot server this made the impacted job details pretty unusable. - So this has shown that it's important for the TestBot to protect itself against such issues. https://bugs.winehq.org/show_bug.cgi?id=48031 - This is also a prerequisite to letting developers set $WINEDEBUG on their jobs: https://bugs.winehq.org/show_bug.cgi?id=47997
* Fix error handling in the WineRun* scripts The current situation makes it hard to highlight the testbot.log errors on the job details page. https://bugs.winehq.org/show_bug.cgi?id=48658
* Add support for com program tests (e.g. chcp.com). https://bugs.winehq.org/show_bug.cgi?id=48090 -> The WTBS has a test case.