https://bugs.winehq.org/show_bug.cgi?id=48655
Bug ID: 48655 Summary: Automate checking the WTBS emails Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs@winehq.org Reporter: fgouget@codeweavers.com Distribution: ---
See bug 48653 for a description of the WTBS and how to check the jobs created when running it on the TestBot.
When a 'wine-devel' job completes the TestBot sends one or two emails. * One of the emails simply sends the task reports to the patch author. * The other is only sent if a task generated new errors.
The point of the TestWTBS.emails script would be to check that:
1. The TestBot sent the task report email for every Wine patch and not for any patch email. This would be based off of the WTBS.Patch.Disposition field.
2. The TestBot sent a 'found new errors' email only when new errors are expected. This would be based off of the 'WTBS.*.NewFailures' property or the expected errors list.
However for both these checks the script would need access to the emails sent by the TestBot. The simplest option would be for the tester to set the WinePatchToOverride property and save the emails to a separate mbox and give the patch to that mbox to the script.
As in the TestWTBS.patches case, the script would only care about a few lines:
* The 'From' lines to delimit the emails.
* The 'Subject' lines to match them to TestBot jobs and thus to the set of WTBS properties. Note that this would not rely on the 'TestBot job xxx results:' part since the new failures email are lacking it. Instead it would match the rest of the subject to the job's Remarks field.
* The 'I think I found new failures' line to identify emails reporting new errors.
The script could later be expanded to check the content of the email or attachments more in details if needed.