On Mon, 3 Jun 2019, Jacek Caban wrote: [...]
Here is how I imagine TestBot could look do:
Accept mingw-w64 patches (like it currently does for Wine patches)
Build a compiler toolchain
This is scripted in many places, so it shouldn't be too hard to automate. We could probably start with the most common configuration.
Francois, how does that sound?
That's probably doable but it will require quite a few TestBot modifications.
* The TestBot will need to know how to deal with multiple repositories. We need that for Wine anyway and I even have some incomplete patches for it.
* It will need to be able to identify whether a patch is against Wine or mingw-w64 which is related to the previous point.
* The TestBot normally analyses the test results to identify failures. That's in part necessary for Wine to distinguish new test failures from old ones. Presumably mingw-w64 has tests that actually pass so simply bypassing these checks and just looking at the process exit code would be enough.
* We may want separate pages for Wine's jobs and the mingw-w64 ones.
* The job submission page would likely need to offer different options for mingw-w64 patches than for Wine ones. For instance if we run mingw-w64 on Windows, those Windows VMs would need to have mingw-w64 installed which the Wine test VMs should not have. So we'd want to offer testing on a different set of VMs for mingw-w64 patches that for Wine ones.
That's one thing that separates the TestBot from other CI systems: it's not generic so where other CI systems just need to be configured for a new project, to some extent, the TestBot needs to be patched through and through.
- Build a few projects - to make sure they build.
Wine, Wine Gecko, VLC should give a good coverage.
That sounds like long running tasks so we will likely need new hardware so the TestBot can run them for every patch that gets submitted: if patches come at a rate of about one per hour, then the TestBot needs to process the resulting jobs at a rate of at least one per hour. That said we already need new hardware for PCI-passthrough for Wine (though the exact details are not set yet).