[Bug 58597] New: AppDB rating system needs to be automated

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=58597

Bug ID: 58597
           Summary: AppDB rating system needs to be automated
           Product: WineHQ Apps Database
           Version: unspecified
          Hardware: x86-64
                OS: Linux
            Status: NEW
          Keywords: download, source
          Severity: enhancement
          Priority: P2
         Component: appdb-unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

Allowing users to select a rating (Bronze, Silver, Gold, etc.) clearly hasn't
worked. Applications are marked Gold and Platinum because the user likes it,
not giving any useful feedback about how capable Wine is to run that
application. Maintainers rarely do anything to improve this.

A rewrite, which won't be quick or easy, allowing the AppDB itself to select a
relevant rating would make more sense.

1. Does the app run? No --> Garbage. Yes --> Continue.

This is just a silly, simple, quick example. I am not suggesting a
"wizard"-style as that would be horrible.

I would also suggest that the rating is calculated (Javascript, for example)
before it is added to the database, probably in the current rating field,
hidden, so that the server isn't given more to do.

This should be doable without having to change anything on the database.

[Bug 58597] AppDB rating system needs to be automated

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=58597

Stian Low  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #1 from Stian Low  ---
(In reply to Ken Sharp from comment #0)
> Allowing users to select a rating (Bronze, Silver, Gold, etc.) clearly
> hasn't worked.

Agreed.  Very hit-or-miss among AppDB ratings and often requires reading
details further to determine a more accurate rating.
> allowing the AppDB itself to select a relevant rating would make more sense.
> - Does the app run? No --> Garbage. Yes --> Continue.

Does the app have bugs? may be more clear to submitters and encourages bug
reports while adding test results.

Linking AppDB ratings to Bugzilla ratings like severity, etc. may offer some
solutions.

Simple example: Garbage (Critical bugs) -> Gold (Normal bugs) -> Plat (No bugs)

But then raises: How to improve reliability of Bugzilla severity ratings?

[Bug 58597] AppDB rating system needs to be automated

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=58597

--- Comment #2 from Ken Sharp  ---
Bugzilla already has severity ratings:
https://bugs.winehq.org/page.cgi?id=fields.html#severity

I don't think linking the bug severity to the AppDB makes any sense, as even a
major bug can have a workaround and this is only evident in the comments. Any
critical or blocker bug potentially affects every application and linking such
a bug to every application on the AppDB would be a bad idea.

I'm happy to take a shot at this when I have enough time to sit down and look
at it properly, but there are more urgent outstanding issues.

[Bug 58597] AppDB rating system needs to be automated

[WineHQ Bugzilla]

http://bugs.winehq.org/show_bug.cgi?id=58597

--- Comment #3 from Stian Low  ---
(In reply to Ken Sharp from comment #2)
> Bugzilla already has severity ratings:
> https://bugs.winehq.org/page.cgi?id=fields.html#severity
>
> I don't think linking the bug severity to the AppDB makes any sense

Agreed, linking AppDB per severity or even simply on bug basis is not
sufficient.  Rather, linking on a per bug comment basis seems more sufficient.
> even a major bug can have a workaround and this is only evident in the comments.
> Any critical or blocker bug potentially affects every application and
> linking such a bug to every application on the AppDB would be a bad idea.

Agreed.  Such critical bugs widely affecting AppDB entries do not need to spawn
efforts to seek out all AppDB entries affected.  However, when tests fail due
to such critical bugs and comments are added regarding the failure, it seems to
make sense at that point to link the failures as test results to AppDB to
reflect the most recent status/rating.

Test results without Platinum highest ratings should link to newly spawned
and/or existing bugs.  Similarly many bug comments should link to newly spawned
and/or existing AppDB entries.
> this is only evident in the comments.

AppDB test results are often partially submitted as bug screenshots, logs,
comments, etc but often not reflected in AppDB because it may be too time
consuming and partially redundant.

Adding AppDB support for links to Bugzilla user comments may offer a solution
to redundancy.

Balancing flexibility with convenience and effectiveness is a challenge.

For flexibility, AppDB maintainers may want to submit test results on behalf of
various Bugzilla users and comments.

A better solution may be encouraging Bugzilla users to submit test results on
their own behalf which may require adding conveniences to Bugzilla to link
comments to AppDB test results.

How may AppDB test result linking be convenient for Bugzilla commenters?  Again
flexibility vs convenience is a challenge.

AppDB linking per bug comment may be necessary to support linking comments to
apps other than the app reported by a bug.  For example, games other than the
game for which a bug is reported may be tested to troubleshoot different
behaviors.  In this case, Bugzilla comments need additional fields that support
selecting from test results listings from AppDB.

Bugzilla may need to query AppDB for test results listing to pick from in a
similar way that AppDB was changed to query Bugzilla for bug links and Wine
versions.

However, doing so partially defeats the purpose of the AppDB and Bugzilla DB
separation recently merged because it adds back more traffic dependence back
between the apps:
https://gitlab.winehq.org/winehq/appdb/-/merge_requests/13

Unlike AppDB which queries Bugzilla only for a few HTML pages, Bugzilla may
query AppDB on a per comment basis.  At that point, consolidation between the
AppDB and Bugzilla seems in order.

Separate user accounts between Bugzilla and AppDB is already a redundancy that
may discourage AppDB engagement.

Perhaps Gitlab which is under consideration to replace Bugzilla offers a
solution to redundancies.

Test results are related to continuous integration which Gitlab offers
supporting features which may be leveragable to provide a more convenient
solution for connecting test results to bugs and comments.  Then AppDB could
behave more as a simple front end similar to:
https://www.protondb.com/

This however may introduce yet another neglected inconvenience with the
overhead of having to maintain comment test result accuracy.  For instance, a
test result for an app may be submitted as garbage against said critical bug
but not retested after the critical bug is resolve so AppDB reflects a garbage
despite a fix.  Ideally the tester would retest to override the previous but
that may not happen.

This inaccuracy however is still a problem with AppDB regardless which
partially prompted this bug report.