https://bugs.winehq.org/show_bug.cgi?id=47904
Bug ID: 47904 Summary: Jobs should also report a count of failed test units Product: Wine-Testbot Version: unspecified Hardware: x86 OS: Linux Status: NEW Severity: normal Priority: P2 Component: unknown Assignee: wine-bugs@winehq.org Reporter: fgouget@codeweavers.com Distribution: ---
Currently jobs report the number of individual ok() failures. That's ok when testing an individual test unit.
But for WineTest jobs one can get numbers like 390 or even 2733. That's impossible to compare to the results of test.winehq.org which report the number of failed test units. In particular test.winehq.org rejects reports that have more than 50 or 70 failed test units and its impossible to tell by looking at the results on the TestBot whether the report was rejected or not.
So it would be nice to report the number of failed test units in addition to the number of individual ok() failures.
This would also be nice when a patch modifies a Wine module so that all its test units are re-run.