[Bug 48216] New: Add a WineTest size, time and test failures comparison page to the TestBot

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48216

Bug ID: 48216
           Summary: Add a WineTest size, time and test failures comparison
                    page to the TestBot
           Product: Wine-Testbot
           Version: unspecified
          Hardware: x86
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

This would expend on the data collected for bug 48215.

This would be a new page that lets the user pick two (or three or more)
WineTest results and put them in the same size, run time and test unit failure
tables as the JobDetails page would if they were all from the same job.

For each column the user would pick a CommitId and then get a list of the
available task reports (for instance "wxppro (32 bit)", "debian10 (32 bit
Chinese:China)").

This would allow comparing a VM's 32 and 64 bit results (these run in separate
jobs). Or compare a VM's latest result against its past result(s). Or compare
the results of running the tests on Windows 8 with those of Windows 10 on the
same hardware. Or comparing a WineTest run that timed out with one that did
not.

In effect this would be similar to bug 15477 but implemented in the TestBot
rather than the on test.winehq.org, the advantage being that the TestBot has
the infrastructure needed to dynamically build pages.

One approach to implement this would be to pour all the report summary
information into a big table. Something like (JobId, StepNo, TaskNo, CommitId,
TestUnit, Failures, Size, RunTime). However given the performance issues with
our current Perl ORM the performance may well be unacceptably slow (see bug
45023).

Another approach would be to use the task-level summary files (described in bug
48215) and use a much smaller table to map commit ids to the tasks with
relevant results. So this table would just contain (JobId, StepNo, TaskNo,
CommitId) and would be used to know which tasks have results for a given commit
id, and once the tasks have been selected, where to find the summary file
containing the data to put in the tables.