[Bug 19377] New: Calculation of overall rating is misleading to users
http://bugs.winehq.org/show_bug.cgi?id=19377 Summary: Calculation of overall rating is misleading to users Product: WineHQ Apps Database Version: unspecified Platform: PC OS/Version: Linux Status: UNCONFIRMED Severity: normal Priority: P2 Component: appdb-unknown AssignedTo: wine-bugs(a)winehq.org ReportedBy: dimesio(a)earthlink.net AFAICT, the overall rating for an app is calculated as an average of the ratings submitted in tests for the most recent version. This leads to overall ratings that do not match any of the submitted tests for that version. The Microsoft Office 2007 installer is a good example: Tests submitted on July 6 and 9 rated it platinum for 1.1.25, and the overall rating was (correctly) platinum. A test submitted on July 14 rated it garbage for 1.1.25, and the overall rating dropped to gold. A test submitted on July 16 rated it garbage for 1.1.25, and the overall rating dropped to silver. This is misleading. The gold rating tells users they need one or more tweaks to get the app working, the silver rating tells them that basic functions work but some advanced ones don't; in the Office 2007 example, neither rating reflected the data that was actually submitted for that version. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=19377 --- Comment #1 from Rosanne DiMesio <dimesio(a)earthlink.net> 2009-07-20 11:23:48 --- Looking at the entry for Powerpoint 2007, it appears I was wrong about the overall rating being an average of recent tests. At this moment, the overall rating for Powerpoint is silver, based on a test submitted on Feb. 27 for 1.1.15, despite the fact that I (the maintainer) submitted a bronze rating for 1.1.22 on May 28. So whatever the formula is, it is not only misleading, but appears to be intentionally so. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=19377 Alexander Nicolaysen Sørnes <alex(a)thehandofagony.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |NEW CC| |alex(a)thehandofagony.com Ever Confirmed|0 |1 --- Comment #2 from Alexander Nicolaysen Sørnes <alex(a)thehandofagony.com> 2009-07-21 10:41:15 --- If there is only one test result for a given Wine release, the Magic Formula tries to pick the Wine version with the best average rating, or (preferably) a Wine release with more than one test report. This is done to avoid showing a 'false negative' due to users having a misconfigured Wine version, bad graphics card etc. I guess we could change the formula to discard reports with a garbase ratings if the same Wine version has been rated higher by another user. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=19377 --- Comment #3 from Rosanne DiMesio <dimesio(a)earthlink.net> 2009-07-21 11:38:24 --- I understand the need to adjust for false garbage reports, but the problem is with using any sort of average at all. One gold and one bronze does not equal a silver because of the way in which these levels are defined, regardless of the accuracy of the individual test reports. If there were hundreds of test reports being submitted for each app for each version of Wine, an average rating might approach some degree of accuracy, but that's not likely to happen. My suggestion would be to abandon the idea of an overall rating altogether and simply display the most recent test report for the latest Wine version, with perhaps preference given to reports submitted by maintainers. But I imagine I'm alone in that opinion. -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.winehq.org/show_bug.cgi?id=19377 Rosanne DiMesio <dimesio(a)earthlink.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mywine(a)schiermeier-software | |.de --- Comment #4 from Rosanne DiMesio <dimesio(a)earthlink.net> 2012-01-23 11:12:24 CST --- *** Bug 29640 has been marked as a duplicate of this bug. *** -- Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email Do not reply to this email, post in Bugzilla using the above URL to reply. ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=19377 --- Comment #5 from Rosanne DiMesio <dimesio(a)earthlink.net> --- Still an issue, and I still believe using any formula to manipulate the overall rating is inherently dishonest. False platinums are a bigger problem than false garbage ratings. Does the formula compensate for that? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=19377 --- Comment #6 from Vincent Povirk <madewokherd(a)gmail.com> --- What if we used the median instead of the average? That way outliers are just ignored. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=19377 --- Comment #7 from Rosanne DiMesio <dimesio(a)earthlink.net> --- Often the outliers are correct. Ratings are specific to Wine versions, and very few apps get more than one test report for any given Wine version. An app can legitimately go from platinum to garbage due to a regression, and back up again to platinum when the regression is fixed. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=19377 Rosanne DiMesio <dimesio(a)earthlink.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jhansonxi(a)gmail.com --- Comment #8 from Rosanne DiMesio <dimesio(a)earthlink.net> --- *** Bug 42088 has been marked as a duplicate of this bug. *** -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=19377 Rosanne DiMesio <dimesio(a)earthlink.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|NEW |RESOLVED Fixed by SHA1| |d96133072376c633f3f825b731b | |ced7703b4af2d --- Comment #9 from Rosanne DiMesio <dimesio(a)earthlink.net> --- The version rating has been changed to simply display the latest rating/tested release. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=19377 Rosanne DiMesio <dimesio(a)earthlink.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |CLOSED --- Comment #10 from Rosanne DiMesio <dimesio(a)earthlink.net> --- Closing fixed AppDB bug. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
participants (1)
-
wine-bugs＠winehq.org