I have done a lot of thinking about the Application Database http://appdb.codeweavers.com lately and how the rating system works. We currently have 848 Applications in there but only 58 with a 5 star rating which works out to less than 10% and thats the good news. The bad news is that of those 58 only 26 rate that without windows installed. After takeing a closer look, none of the apps I looked at actualy rated 5 star by my way of judging. Now maybe I'm overly critical but in my way of thinking If the app doesn't install in wine then it shouldn't get a 5. IMHO We need to set criteria that give us a better/less subjective way of measuring how well we are doing. So this is what I have come up with.
1. Installation (total possible 2) Installs in wine with fake windows 2 points Installs in wine with windows mounted 1 point Does not install in wine 0 points
2. Running & DLLS (total possible 2) Runs fine using all builtin dlls 2 points Runs fine using native dlls 1 point Does not run/crashes on startup 0 points
3 Running general (total possible 2) program does not crash ever 2 points Program crashes only after long use or rare aspect used 1 point program crashes on startup or doing common function 0 points
4. Screen Presentation/layout (total possible 2) App looks exactly like it should 2 points Small problems that do not affect working with app 1 point App is unusable because of display problems 0 points
5. Fonts (total possible 2) Fonts look exacty right 2 points Fonts not exactly right but acceptable 1 point Fonts unreadable or totaly wrong. 0 points
6. Mouse (total possible 2) Mouse behaves as expected 2 points Mouse behaves strange but can still use app 1 point Mouse behaves badly and is unusable 0 points
7. Keyboard (total possible 2) Keyboard reacts as expected. 2 points Some keys/key combinations dont work but most do. 1 point Keyboard locks up or unresposive 0 points
8. Files save/restore (total possible 2) Able to save and restore files without any problems 2 points Can save and restore files but have problems 1 points Cannot save or restore files 0 points
9. Sound (total possible 2) Not applicable Sound is correct 2 points sound is ok most of the time but has minor glichs 1 point Sound is garbled or missing 0 points
10.Joystick/gamepad (total possible 2) Not applicable joystick/gamepad works correctly 2 points joystick/gamepad have problems but mostly work 1 point joystick/gamepad unresponsive 0 points
11.Registy/rememberd state (total possible 2) Not applicable Program remembers key settings from one session to next 2 points Program remembers most settings but not all 1 point program does not remember most settings at all 0 points
12. Date tested. Verison of wine used
Each of the sections should have a comment field and a place to enter a url to a bug report if applicable
Applications could be scored based as a percentage. Applications that use all critera would need all 22 points to get a 100% rating. For other applications some of these criteria are not applicable and their score would use a lower base score.
Idealy we would be able to look at the AppDB and list all programs that have problems with any of the criteria
Probably the most usefull aspect of this is that it gives users and developers a real way of measuring how well wine really is doing and where problems that affect the most programs are.
I have cross posted this because I think this is important to both users and developers. Any comments, criticisims or helpfull suggestions welcome
Tony Lambregts
An excellent idea!
--- Tony Lambregts tony_lambregts@telusplanet.net wrote:
- Date tested. Verison of wine used
Version of operating system? Hardware configuration?
For other applications some of these criteria are
not
applicable and their score would use a lower base score.
Would it be easier to just give the app the full points for that category? So "fully works or not-applicable" is two points, "sort of works" is one point, and "broke" is zero.
===== Jon Miner http://www.ConcordNH.com/Trains - Concord Model RR Club http://bmrrhs.come.to - B&M RR Historical Society
__________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com
I Agree that this is a great idea.
I always felt that the appdb left much to be desired.
Kevin
===== --------------------------------- Kevin DeKorte kdekorte@yahoo.com
__________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com
I also agree that this is a great idea and that the application database needs to be changed to more accurately reflect the state of wine.
I'm still of the opinion that the applications matter the most.
How can we get these changes implemented? How can we integrate our old data with this new format? We don't want to lose all the data we already have.
I also agree that we need os/platform/window manager data and that n/a should be a 2, so we can one common score for complete applications. That will make the data much easier to understand.
On Thu, May 02, 2002 at 09:53:15PM -0600, Tony Lambregts wrote:
I have done a lot of thinking about the Application Database http://appdb.codeweavers.com lately and how the rating system works. We currently have 848 Applications in there but only 58 with a 5 star rating which works out to less than 10% and thats the good news. The bad news is that of those 58 only 26 rate that without windows installed.
On Thu, May 02, 2002 at 09:53:15PM -0600, Tony Lambregts wrote:
I have done a lot of thinking about the Application Database http://appdb.codeweavers.com lately and how the rating system works. We currently have 848 Applications in there but only 58 with a 5 star rating which works out to less than 10% and thats the good news. The bad news is that of those 58 only 26 rate that without windows installed. After takeing a closer look, none of the apps I looked at actualy rated 5 star by my way of judging. Now maybe I'm overly critical but in my way of thinking If the app doesn't install in wine then it shouldn't get a 5. IMHO We need to set criteria that give us a better/less subjective way of measuring how well we are doing. So this is what I have come up with.
Installation (total possible 2) Installs in wine with fake windows 2 points Installs in wine with windows mounted 1 point Does not install in wine 0 points
Running & DLLS (total possible 2) Runs fine using all builtin dlls 2 points Runs fine using native dlls 1 point Does not run/crashes on startup 0 points
3 Running general (total possible 2) program does not crash ever 2 points Program crashes only after long use or rare aspect used 1 point program crashes on startup or doing common function 0 points
Screen Presentation/layout (total possible 2) App looks exactly like it should 2 points Small problems that do not affect working with app 1 point App is unusable because of display problems 0 points
Fonts (total possible 2) Fonts look exacty right 2 points Fonts not exactly right but acceptable 1 point Fonts unreadable or totaly wrong. 0 points
Mouse (total possible 2) Mouse behaves as expected 2 points Mouse behaves strange but can still use app 1 point Mouse behaves badly and is unusable 0 points
Keyboard (total possible 2) Keyboard reacts as expected. 2 points Some keys/key combinations dont work but most do. 1 point Keyboard locks up or unresposive 0 points
Files save/restore (total possible 2) Able to save and restore files without any problems 2 points Can save and restore files but have problems 1 points Cannot save or restore files 0 points
Sound (total possible 2) Not applicable Sound is correct 2 points sound is ok most of the time but has minor glichs 1 point Sound is garbled or missing 0 points
10.Joystick/gamepad (total possible 2) Not applicable joystick/gamepad works correctly 2 points joystick/gamepad have problems but mostly work 1 point joystick/gamepad unresponsive 0 points
11.Registy/rememberd state (total possible 2) Not applicable Program remembers key settings from one session to next 2 points Program remembers most settings but not all 1 point program does not remember most settings at all 0 points
- Date tested. Verison of wine used
Way too complicated IMHO.
Do you really think people will put up with this system if currently they very frequently don't even mention whether they actually tried the app on Wine and what results they got ? (yes, I added a comment which asks them to also mention this)
A lot of the submissions that I commit leave several things to be desired, too (which I frequently improve a lot before committing, which shouldn't really be my job, though).
IMHO the fact that the current *easy* rating system doesn't get used properly sort of indicates that it would be even worse with a more complicated system. One possible bad result: less people putting up with submitting apps, because it's "too much work", thus dramatically fewer apps listed. Do we want that ?
IMHO we want as many apps listed as possible, even with a somewhat fuzzy picture, as opposed to simply listing Office, IE, HalfLife and Starcraft with perfect rating and description.
People do want to know whether a particular highly weird app (take "Femta" as an example ;-) might have the slightest chance of running on Wine, so having a relatively broad base is also pretty important.
Andreas Mohr wrote:
Way too complicated IMHO.
Do you really think people will put up with this system if currently they very frequently don't even mention whether they actually tried the app on Wine and what results they got ? (yes, I added a comment which asks them to also mention this)
A lot of the submissions that I commit leave several things to be desired, too (which I frequently improve a lot before committing, which shouldn't really be my job, though).
IMHO the fact that the current *easy* rating system doesn't get used properly sort of indicates that it would be even worse with a more complicated system. One possible bad result: less people putting up with submitting apps, because it's "too much work", thus dramatically fewer apps listed. Do we want that ?
IMHO we want as many apps listed as possible, even with a somewhat fuzzy picture, as opposed to simply listing Office, IE, HalfLife and Starcraft with perfect rating and description.
People do want to know whether a particular highly weird app (take "Femta" as an example ;-) might have the slightest chance of running on Wine, so having a relatively broad base is also pretty important.
Of course its more complicated. The only response that I have to this is my real life experience with my other hobbie LEGO. Over the years LEGO has put out thousands of sets. About a year and a half ago someone decided that it would be nice to have a database of what all these sets contained with cross references for all the pieces. Over time people have added to the database. In some cases it _just_ involed opening the package and counting the number of each type of piece, getting the right part number of each type of piece, entering the inventory into a text file and submitting it by email. In some (a lot) cases it involved building the model from pieces first. You can check it out at http://peeron.com/inv
Rating the apps is not the same as submitting the apps and of course some people could not be bothered. However having a good rating system would encourage people to use it. There are people like me that will go to the kind of effort needed to build this properly if it is set up. A lot of the programs are freely downloadable or have a demo. People who do not have the skill level to be developers would have a way to contribute. It is amazing what people will do if it is important to them.
The question is this: Would the rating system that I have proposed be usefull to users and developers? For me the answer is yes and I say "If you build it they will come".
Tony Lambregts
Tony Lambregts
--- Andreas Mohr andi@rhlx01.fht-esslingen.de wrote: [... skipped ...]
Do you really think people will put up with this system if currently they very frequently don't even mention whether they actually tried the app on Wine and what results they got ? (yes, I added a comment which asks them to also mention this)
A lot of the submissions that I commit leave several things to be desired, too (which I frequently improve a lot before committing, which shouldn't really be my job, though).
[... skipped ...]
I think the main problem of AppDB is lack of contributions. Comments from Andreas confirm that. There are no volunteers who would spend time testing the application, submitting the bugs to bugzilla, making time to time regression testing.
If we implement the rating subsystem it can attract new contributors, but I doubt. Somebody has to go and spend time testing the applications. I don't want Andreas to be this person - it's a waste to spend his time on this job without significant effects. I'd better assign a couple bugs I can't handle to him (I'm the first in the line ;-).
The way of improving the application DB I see is to increase soliciting for help, find volunteers, who'd take ownership over information about the applications, give the application owners access to edit AppDB information. If somebody asks - No, I do not want to take ownership over any applications. Why? Because I do not run any Windows applications, I have enough native ones.
Another point - there are already commercial products, developed by companies, contributing to Wine. You, guys already have list of applications your products support, list applications you want to support, you already have testing, regression testing. Why not take ownership over applications you are interested in? Publishing information about application status in AppDB you can increase number of bug reports and won't take much time.
One more thing - I'd like to remove discussions from AppDB completely and leave only static information there. The proper place for the discussions are Bugzilla and wine-users. Discussions become outdated very quickly and can't be corrected by application owners. Even requests to change information, published in AppDB should be made through Bugzilla.
Andriy
__________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com
I know of a few wine users, the lindows insiders. ;)
We get tons of responses about application compability, but our current data is not very useful because it pertains to wine from cvs from 11/01.
After our next release, I'll work on a system to get our insider application feedback to your application database.
On Fri, May 03, 2002 at 11:24:38AM -0700, Andriy Palamarchuk wrote:
--- Andreas Mohr andi@rhlx01.fht-esslingen.de wrote: [... skipped ...]
Do you really think people will put up with this system if currently they very frequently don't even mention whether they actually tried the app on Wine and what results they got ? (yes, I added a comment which asks them to also mention this)
A lot of the submissions that I commit leave several things to be desired, too (which I frequently improve a lot before committing, which shouldn't really be my job, though).
[... skipped ...]
I think the main problem of AppDB is lack of contributions. Comments from Andreas confirm that. There are no volunteers who would spend time testing the application, submitting the bugs to bugzilla, making time to time regression testing.
If we implement the rating subsystem it can attract new contributors, but I doubt. Somebody has to go and spend time testing the applications. I don't want Andreas to be this person - it's a waste to spend his time on this job without significant effects. I'd better assign a couple bugs I can't handle to him (I'm the first in the line ;-).
--- Michael Cardenas michael.cardenas@lindows.com wrote:
I know of a few wine users, the lindows insiders. ;)
We get tons of responses about application compability, but our current data is not very useful because it pertains to wine from cvs from 11/01.
After our next release, I'll work on a system to get our insider application feedback to your application database.
Thank you Michael. It would be great to have updated infromation about current status of the application in AppDB.
Andriy
__________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com
On Fri, 2002-05-03 at 11:39, Andreas Mohr wrote:
On Thu, May 02, 2002 at 09:53:15PM -0600, Tony Lambregts wrote:
I have done a lot of thinking about the Application Database http://appdb.codeweavers.com lately and how the rating system works. We
<snip>
Way too complicated IMHO.
<snip>
People do want to know whether a particular highly weird app (take "Femta" as an example ;-) might have the slightest chance of running on Wine, so having a relatively broad base is also pretty important.
I agree with Andi. Too much useless, redundant data. If you have an issue with Wine, submit it to Bugzilla (bugs.winehq.com). Bugzilla is where Wine bugs belong. The AppDB has to only answer a few simple questions. "Does my app work?" and "How well does my app work?".
I do feel the AppDB does need a better rating system though. My plan is to add a Wine version table to the Ratings. You would be able to Rate an Application with 0-5 stars, pick the Version you where using, and if you where using a Windows partition or a Fake Windows install. Also, I think that each rating submission, should get a comment and a timestamp. This would allow us to keep up-to-date info.
This addition, brings me to my second AppDB addition. To add a reports section. The reports section would contain reports for: - 20 highest rated Apps. (based on a calculation to be decided later) - 20 highest voted Apps. (I plan to make the voting system more visable) - Newest App Additions to the DB. (by popular request) - Most Active Apps (by number of comments/ratings)
I also saw Andriy's comments about removing discussions. I oppose this strongly. But, we can put in an option to hide them for those that do not wish to read them. For us that like the comments, I think the comments needs a sort toggle. I like to see the comments sorted by newest first. Right now, oldest messages are at the top of the list.
--- Jeremy Newman jnewman@codeweavers.com wrote:
I also saw Andriy's comments about removing discussions. I oppose this strongly. But, we can put in an option to hide them for those that do not wish to read them. For us that like the comments, I think the comments needs a sort toggle. I like to see the comments sorted by newest first. Right now, oldest messages are at the top of the list.
No need to hide comments if they are there. The sorting order change is a good idea.
Andriy
__________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com
I agree with Andi. Too much useless, redundant data. If you have an issue with Wine, submit it to Bugzilla (bugs.winehq.com). Bugzilla is where Wine bugs belong. The AppDB has to only answer a few simple questions. "Does my app work?" and "How well does my app work?".
I also agree strongly. The old app db failed in part, imo, because it tried to gather too much data.
I wanted to bring up one enhancement that many people I've talked to feel would be a very important improvement to the appdb: get more 'Lawsons'.
The idea is if we can get someone to do for each app in the appdb what Lawson does for Juno, then I believe that Wine will truly start to fly.
So, I guess the concept is sort of an 'owner', or maybe a team of owners for each app. An ideal owner would be someone who: a) Uses the app everyday. b) Uses the app fairly broadly and depends on it. c) Is willing to periodically try CVS builds of Wine and to report regressions.
Further, if we can get an owner for each app who actually uses it, then I think that person would get the opportunity to rate it, and their rating would weigh more heavily than all the other comments.
Jer
Well, now that we're up to date with the public tree, I've been trying to be the "Lawson" for office.
That's a hefty job though, so I'm enlisting the help of our qa dept in entering bugs into bugs.winehq.com.
On Fri, May 03, 2002 at 02:47:31PM -0500, Jeremy White wrote:
I agree with Andi. Too much useless, redundant data. If you have an issue with Wine, submit it to Bugzilla (bugs.winehq.com). Bugzilla is where Wine bugs belong. The AppDB has to only answer a few simple questions. "Does my app work?" and "How well does my app work?".
...
I wanted to bring up one enhancement that many people I've talked to feel would be a very important improvement to the appdb: get more 'Lawsons'.
The idea is if we can get someone to do for each app in the appdb what Lawson does for Juno, then I believe that Wine will truly start to fly.
So, I guess the concept is sort of an 'owner', or maybe a team of owners for each app. An ideal owner would be someone who: a) Uses the app everyday. b) Uses the app fairly broadly and depends on it. c) Is willing to periodically try CVS builds of Wine and to report regressions.
Further, if we can get an owner for each app who actually uses it, then I think that person would get the opportunity to rate it, and their rating would weigh more heavily than all the other comments.
Jer
Jeremy White wrote:
I agree with Andi. Too much useless, redundant data. If you have an issue with Wine, submit it to Bugzilla (bugs.winehq.com). Bugzilla is where Wine bugs belong. The AppDB has to only answer a few simple questions. "Does my app work?" and "How well does my app work?".
I also agree strongly. The old app db failed in part, imo, because it tried to gather too much data.
I wanted to bring up one enhancement that many people I've talked to feel would be a very important improvement to the appdb: get more 'Lawsons'.
The idea is if we can get someone to do for each app in the appdb what Lawson does for Juno, then I believe that Wine will truly start to fly.
So, I guess the concept is sort of an 'owner', or maybe a team of owners for each app. An ideal owner would be someone who: a) Uses the app everyday. b) Uses the app fairly broadly and depends on it. c) Is willing to periodically try CVS builds of Wine and to report regressions.
Further, if we can get an owner for each app who actually uses it, then I think that person would get the opportunity to rate it, and their rating would weigh more heavily than all the other comments.
This is exactly what I am talking about but we are coming at it from different directions. The way I see it the rating page would be the responsability of the app owners. I don't want to rate MS Word because I dont use it. I can rate the apps that I use and the amount of work to do this right is small. I would be willing to"adopt" one or two apps if they were important enough. Maybe others would feel the same way. The reason for having this is so that we have a consistant criteria from one app to the next. I don't want a hundred users giving an opinion. I want a way of measuring/quantifying how we are doing.
Tony Lambregts
On Fri, May 03, 2002 at 02:15:25PM -0500, Jeremy Newman wrote:
I also saw Andriy's comments about removing discussions. I oppose this strongly. But, we can put in an option to hide them for those that do not wish to read them. For us that like the comments, I think the comments needs a sort toggle. I like to see the comments sorted by newest first. Right now, oldest messages are at the top of the list.
Yep, same here, I also oppose to this strongly. The comments are the stuff that *really* matters. There are a LOT of useful hints in there that would *never* get written in the AppDB if it weren't for the fact that you can simply babble something in a comment instead of telling someone to maybe add this info for you in 3 weeks or so. Now if only you could actually combine the content of comments back into the main entry and then *delete* those now useless comments, that'd be the only thing missing here in order to be able to create a really good HOWTO for an app. That would be the job of a maintainer for a specific app, of course.
On Thu, 2 May 2002, Tony Lambregts wrote:
So this is what I have come up with.
- Installation (total possible 2)
[...]
- Date tested. Verison of wine used
With the proviso that having multiple categories tends to favour "average-looking" results, the system looks good.
I would also include the license under which the software is available. Without wishing to reigniting the wine-license debate :) the license could be one of: "free w/ sourcecode", "free but binary only", "commercial", "Donno, got it off a mate".
Idealy we would be able to look at the AppDB and list all programs that have problems with any of the criteria
Yes.
Also, it would be nice if you could search the db for applications that have a certain range of points (or stars), rather than just >= x, for example to look for all four-star applications.
Probably the most usefull aspect of this is that it gives users and developers a real way of measuring how well wine really is doing and where problems that affect the most programs are.
Hmm, there was always the possibility of this, but I think any half-way reliable estimate would require retesting working apps; or do you assume no regression in functionality?
What do you think?
---- Paul Millar
-
Andreas Mohr -
Andriy Palamarchuk -
Jeremy Newman -
Jeremy White -
Jon Miner -
Kevin DeKorte -
Michael Cardenas -
Paul Millar -
Tony Lambregts