On Wed, 5 Jun 2019, Francois Gouget wrote:
>> 3. Build a few projects - to make sure they build.
>>
>> Wine, Wine Gecko, VLC should give a good coverage.
>
> That sounds like long running tasks so we will likely need new hardware
> so the TestBot can run them for every patch that gets submitted: if
> patches come at a rate of about one per hour, then the TestBot needs to
> process the resulting jobs at a rate of at least one per hour. That said
> we already need new hardware for PCI-passthrough for Wine (though the
> exact details are not set yet).
Those projects are rather large, so either you need quite beefy hardware,
or you just run some smaller test setup for each patch (rebuild mingw-w64
and build a few smoketests maybe?) and then run a full build of those
projects e.g. nightly (e.g. with a known good pinned version of those
projects, which can be updated regularly). That requires a bit more manual
intervention every time there's a regression though.
For the build bot I used to run, a full build of gcc and its test suite would take several days for the cygwin target and host.
I mention that because in my experience, it's far more important to test against the breakages due to compiler changes rather than the comparatively small things that change day to day on mingw. Very few people on gcc test their changes on w64 (Jonathan Wakely is a recent notable exception), and this tends to cause entire languages to break (see Ada for example).