[Bug 39380] New: test.winehq.org: Provide a way to know why a report was rejected

https://bugs.winehq.org/show_bug.cgi?id=39380

Bug ID: 39380
           Summary: test.winehq.org: Provide a way to know why a report
                    was rejected
           Product: WineHQ.org
           Version: unspecified
          Hardware: x86
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: www-unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

When WineTest submits a report that report can get thrown away. Either because
it's too big, because it has too many failures, because too many tests were
skipped or because of other reasons unknown.

The WineTest.exe is never notified of this because the report analysis is run
asynchronously with respect tot he submission process. But annoyingly there is
no way for the submitter to know why the report was rejected even later,
levaing him to guess as to what needs fixing.

One way to plug this hole would be to send an email with the reason to the
address field in the report. But that could probably be abused to send emails
to arbitrary persons.

A very simple alternative would be to create a per-commit text file containing
one line per ignored report with the report's tag, timestamp of when it was
analyzed and the reason why it was ignored.

[Bug 39380] test.winehq.org: Provide a way to know why a report was rejected

https://bugs.winehq.org/show_bug.cgi?id=39380

--- Comment #1 from François Gouget  ---
The errors can now be seen at the URL below.

https://test.winehq.org/data/errors.html

However:
* There is no link to that page.
* Due to bugs in dissect, a lot of reports can still end up being lost.

[Bug 39380] test.winehq.org: Provide a way to know why a report was rejected

https://bugs.winehq.org/show_bug.cgi?id=39380

François Gouget  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #2 from François Gouget  ---
As far as I can tell there is no bug in dissect that could cause reports to be
lost: every error case goes through mydie() which stores the error for use to
build the errors page.

There are a bunch of (soon to be fixed) cases where dissect ends up
manipulating undefined values and those may cause the report to be put in the
'Other' column. But it's not lost or rejected in that case.

Furthermore the main index page now has a link to the rejected reports page.

So this is all fixed now.

[Bug 39380] test.winehq.org: Provide a way to know why a report was rejected

https://bugs.winehq.org/show_bug.cgi?id=39380

François Gouget  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|[e-mail]        |[e-mail]
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |---

--- Comment #3 from François Gouget  ---
I figured out what happens.

When dissect rejects a report no build directory is marked as outdated. As a
result gather has nothing to do and thus the main index files don't need an
updated. So the cron script skips re-running build-index. However build-index
is also responsible for updating errors.html.

So when a report is rejected one will not know why until another report is
accepted.

[Bug 39380] test.winehq.org: Provide a way to know why a report was rejected

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=39380

François Gouget  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #4 from François Gouget  ---
This is fixed:

commit f7b0a4cc7e3a7f48a4f987ef4907ac753f5ff692
Author:     Francois Gouget 
AuthorDate: Wed May 17 11:27:08 2017 +0200
Commit:     Alexandre Julliard 
CommitDate: Wed May 17 18:41:25 2017 +0200

winetest: Run build-index and/or build-errors as needed.

If a report has been accepted then run both gather and build-index to
    refresh the index files.
    If a report has been rejected run build-errors to update errors.html.
    And if there is nothing new both can be skipped.

Signed-off-by: Francois Gouget 
    Signed-off-by: Alexandre Julliard

[Bug 39380] test.winehq.org: Provide a way to know why a report was rejected

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=39380

Rosanne DiMesio  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #5 from Rosanne DiMesio  ---
Closing fixed website bug.