http://bugs.winehq.org/show_bug.cgi?id=19377
Summary: Calculation of overall rating is misleading to users Product: WineHQ Apps Database Version: unspecified Platform: PC OS/Version: Linux Status: UNCONFIRMED Severity: normal Priority: P2 Component: appdb-unknown AssignedTo: wine-bugs@winehq.org ReportedBy: dimesio@earthlink.net
AFAICT, the overall rating for an app is calculated as an average of the ratings submitted in tests for the most recent version. This leads to overall ratings that do not match any of the submitted tests for that version.
The Microsoft Office 2007 installer is a good example:
Tests submitted on July 6 and 9 rated it platinum for 1.1.25, and the overall rating was (correctly) platinum.
A test submitted on July 14 rated it garbage for 1.1.25, and the overall rating dropped to gold.
A test submitted on July 16 rated it garbage for 1.1.25, and the overall rating dropped to silver.
This is misleading. The gold rating tells users they need one or more tweaks to get the app working, the silver rating tells them that basic functions work but some advanced ones don't; in the Office 2007 example, neither rating reflected the data that was actually submitted for that version.
http://bugs.winehq.org/show_bug.cgi?id=19377
--- Comment #1 from Rosanne DiMesio dimesio@earthlink.net 2009-07-20 11:23:48 --- Looking at the entry for Powerpoint 2007, it appears I was wrong about the overall rating being an average of recent tests.
At this moment, the overall rating for Powerpoint is silver, based on a test submitted on Feb. 27 for 1.1.15, despite the fact that I (the maintainer) submitted a bronze rating for 1.1.22 on May 28.
So whatever the formula is, it is not only misleading, but appears to be intentionally so.
http://bugs.winehq.org/show_bug.cgi?id=19377
Alexander Nicolaysen Sørnes alex@thehandofagony.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |NEW CC| |alex@thehandofagony.com Ever Confirmed|0 |1
--- Comment #2 from Alexander Nicolaysen Sørnes alex@thehandofagony.com 2009-07-21 10:41:15 --- If there is only one test result for a given Wine release, the Magic Formula tries to pick the Wine version with the best average rating, or (preferably) a Wine release with more than one test report. This is done to avoid showing a 'false negative' due to users having a misconfigured Wine version, bad graphics card etc.
I guess we could change the formula to discard reports with a garbase ratings if the same Wine version has been rated higher by another user.
http://bugs.winehq.org/show_bug.cgi?id=19377
--- Comment #3 from Rosanne DiMesio dimesio@earthlink.net 2009-07-21 11:38:24 --- I understand the need to adjust for false garbage reports, but the problem is with using any sort of average at all. One gold and one bronze does not equal a silver because of the way in which these levels are defined, regardless of the accuracy of the individual test reports.
If there were hundreds of test reports being submitted for each app for each version of Wine, an average rating might approach some degree of accuracy, but that's not likely to happen.
My suggestion would be to abandon the idea of an overall rating altogether and simply display the most recent test report for the latest Wine version, with perhaps preference given to reports submitted by maintainers. But I imagine I'm alone in that opinion.
http://bugs.winehq.org/show_bug.cgi?id=19377
Rosanne DiMesio dimesio@earthlink.net changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |mywine@schiermeier-software | |.de
--- Comment #4 from Rosanne DiMesio dimesio@earthlink.net 2012-01-23 11:12:24 CST --- *** Bug 29640 has been marked as a duplicate of this bug. ***
https://bugs.winehq.org/show_bug.cgi?id=19377
--- Comment #5 from Rosanne DiMesio dimesio@earthlink.net --- Still an issue, and I still believe using any formula to manipulate the overall rating is inherently dishonest. False platinums are a bigger problem than false garbage ratings. Does the formula compensate for that?
https://bugs.winehq.org/show_bug.cgi?id=19377
--- Comment #6 from Vincent Povirk madewokherd@gmail.com --- What if we used the median instead of the average? That way outliers are just ignored.
https://bugs.winehq.org/show_bug.cgi?id=19377
--- Comment #7 from Rosanne DiMesio dimesio@earthlink.net --- Often the outliers are correct. Ratings are specific to Wine versions, and very few apps get more than one test report for any given Wine version. An app can legitimately go from platinum to garbage due to a regression, and back up again to platinum when the regression is fixed.
https://bugs.winehq.org/show_bug.cgi?id=19377
Rosanne DiMesio dimesio@earthlink.net changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |jhansonxi@gmail.com
--- Comment #8 from Rosanne DiMesio dimesio@earthlink.net --- *** Bug 42088 has been marked as a duplicate of this bug. ***
https://bugs.winehq.org/show_bug.cgi?id=19377
Rosanne DiMesio dimesio@earthlink.net changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|NEW |RESOLVED Fixed by SHA1| |d96133072376c633f3f825b731b | |ced7703b4af2d
--- Comment #9 from Rosanne DiMesio dimesio@earthlink.net --- The version rating has been changed to simply display the latest rating/tested release.
https://bugs.winehq.org/show_bug.cgi?id=19377
Rosanne DiMesio dimesio@earthlink.net changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |CLOSED
--- Comment #10 from Rosanne DiMesio dimesio@earthlink.net --- Closing fixed AppDB bug.
-
wine-bugs＠winehq.org