Bug triage, or spam?
Hi all, I just have a quick question.. I got a message from someone last night asking me to stop closing bugs because I'm spamming the list, however I also received a message from someone a couple of nights ago thanking me for doing bug triage.. Which is it, and if it is both, then what draws the line between triage and spam, and how do I close stale bugs without it being considered spam? -- Thanks Tom
Can someone explain to me the point of closing bugs please? If it's resolved one way or another then surely that's enough? Just seems like a waste of effort to be honest. Ben H. Tom Spear wrote:
Hi all, I just have a quick question..
I got a message from someone last night asking me to stop closing bugs because I'm spamming the list, however I also received a message from someone a couple of nights ago thanking me for doing bug triage.. Which is it, and if it is both, then what draws the line between triage and spam, and how do I close stale bugs without it being considered spam?
On 5/31/07, Ben Hodgetts <ben(a)atomnet.co.uk> wrote:
Can someone explain to me the point of closing bugs please? If it's resolved one way or another then surely that's enough? Just seems like a waste of effort to be honest.
Well, I can tell you this about it, it's supposed to work like this: - User reports bug - Developer looks into bug, and eventually fixes it, at which point he marks it resolved, and asks user to verify - User says yes it is resolved and marks it verified, or says no it is not, at which point developer reopens bug, and the process repeats - Once marked verified, developer then goes back and closes bug. It hardly ever works that way, unfortunately, but I prefer to see a bunch of closed bugs than ones that are sitting marked resolved as invalid, or resolved as works for me, etc, since invalid isnt really "resolved", and neither is works for me.. Thats why I go thru and close the bugs that are just left resolved for more than a month or 2. -- Thanks Tom
On Thu, May 31, 2007 at 10:04:00AM -0500, Tom Spear wrote:
On 5/31/07, Ben Hodgetts <ben(a)atomnet.co.uk> wrote:
Can someone explain to me the point of closing bugs please? If it's resolved one way or another then surely that's enough? Just seems like a waste of effort to be honest.
Well, I can tell you this about it, it's supposed to work like this:
- User reports bug - Developer looks into bug, and eventually fixes it, at which point he marks it resolved, and asks user to verify - User says yes it is resolved and marks it verified, or says no it is not, at which point developer reopens bug, and the process repeats - Once marked verified, developer then goes back and closes bug.
It hardly ever works that way, unfortunately, but I prefer to see a bunch of closed bugs than ones that are sitting marked resolved as invalid, or resolved as works for me, etc, since invalid isnt really "resolved", and neither is works for me.. Thats why I go thru and close the bugs that are just left resolved for more than a month or 2.
Well, but in theory they could sit at resolved until the end of time ;) But keep it up, people can just delete those RESOLVED->CLOSED transition mails. Ciao, Marcus
On 5/31/07, Tom Spear <speeddymon(a)gmail.com> wrote:
Hi all, I just have a quick question..
I got a message from someone last night asking me to stop closing bugs because I'm spamming the list, however I also received a message from someone a couple of nights ago thanking me for doing bug triage.. Which is it, and if it is both, then what draws the line between triage and spam, and how do I close stale bugs without it being considered spam?
Marking bugs as closed has nothing to do with bug triage. Triaging bugs would be a really helpful thing, but mass-closing bugs does nothing but give subscribers a whole lot of emails to delete. We don't keep track of stats like other projects, so there's really no point to blindly close all bugs that are resolved, instead of closing bugs on a case by case basis.
since invalid isnt really "resolved", and neither is works for me
But closing these bugs somehow makes them "resolved" for you? -- James Hawkins
On 5/31/07, James Hawkins <truiken(a)gmail.com> wrote:
Marking bugs as closed has nothing to do with bug triage. Triaging bugs would be a really helpful thing, but mass-closing bugs does nothing but give subscribers a whole lot of emails to delete. We don't keep track of stats like other projects, so there's really no point to blindly close all bugs that are resolved, instead of closing bugs on a case by case basis.
I'm not blindly closing all bugs marked resolved.. It seems to me that you didnt real thru all of the emails, but I have been posting comments, etc before closing, and there are some that are marked resolved that I did not close, or that I even reopened based on the most recent commentS that indicate the bug might not be fixed, or might actully be a real wine bug..
since invalid isnt really "resolved", and neither is works for me
But closing these bugs somehow makes them "resolved" for you?
It's an organizational thing, if they are closed, then we know for sure that they are resolved. Since everyone is prone to marking a bug as invalid when it is really a bug, or works for me when it really should be open because some users are experiencing it but not others, leaving it as resolved just says that the person who resolved it was too lazy to close it, and that they didn't check back to make sure that some other users didnt have the same problem.. Case in point: bug 3889, has been resolved as worksforme (i did it over a year ago), when it is really a bug, and has never truely been resolved in wine. I just looked back thru the conversation that took place on wine-devel and saw no definitive response saying that the bug was fixed by an alexandre patch, and saw nothing that says the bug is not a bug, so I am about to reopen it and ask if the issue is fixed in current wine. -- Thanks Tom
On 5/31/07, Tom Spear <speeddymon(a)gmail.com> wrote:
On 5/31/07, James Hawkins <truiken(a)gmail.com> wrote:
Marking bugs as closed has nothing to do with bug triage. Triaging bugs would be a really helpful thing, but mass-closing bugs does nothing but give subscribers a whole lot of emails to delete. We don't keep track of stats like other projects, so there's really no point to blindly close all bugs that are resolved, instead of closing bugs on a case by case basis.
I'm not blindly closing all bugs marked resolved.. It seems to me that you didnt real thru all of the emails, but I have been posting comments, etc before closing, and there are some that are marked resolved that I did not close, or that I even reopened based on the most recent commentS that indicate the bug might not be fixed, or might actully be a real wine bug..
I do read through all of your bug emails, which is exactly the problem, because I don't trust that you make the right decision on every bug, and in some cases I've had to go back and correct it. That is the issue.
since invalid isnt really "resolved", and neither is works for me
But closing these bugs somehow makes them "resolved" for you?
It's an organizational thing, if they are closed, then we know for sure that they are resolved.
The statement still stands. Resolved isn't resolved enough for you, but closed is...If a bug is marked resolved, I'm pretty sure it's resolved.
Since everyone is prone to marking a bug as invalid when it is really a bug, or works for me when it really should be open because some users are experiencing it but not others, leaving it as resolved just says that the person who resolved it was too lazy to close it, and that they didn't check back to make sure that some other users didnt have the same problem.. Case in point: bug 3889, has been resolved as worksforme (i did it over a year ago), when it is really a bug, and has never truely been resolved in wine. I just looked back thru the conversation that took place on wine-devel and saw no definitive response saying that the bug was fixed by an alexandre patch, and saw nothing that says the bug is not a bug, so I am about to reopen it and ask if the issue is fixed in current wine.
No one has problems with people correcting bugs that are marked incorrectly, but of the 30 or so bugs a day that you change, this case is a small amount. If you *only* changed this class of bugs, then that would be fine. -- James Hawkins
On 5/31/07, James Hawkins <truiken(a)gmail.com> wrote:
I do read through all of your bug emails, which is exactly the problem, because I don't trust that you make the right decision on every bug, and in some cases I've had to go back and correct it. That is the issue.
It has been a small number of cases..
The statement still stands. Resolved isn't resolved enough for you, but closed is...If a bug is marked resolved, I'm pretty sure it's resolved.
Obviously someone that commits patches to the upstream bugzilla tree agrees with me, because otherwise, there either wouldnt be a resolved option, OR there would be a close option on the bugs that are in any state of open, which is something I have suggested before, to cut down on list traffic.
No one has problems with people correcting bugs that are marked incorrectly, but of the 30 or so bugs a day that you change, this case is a small amount. If you *only* changed this class of bugs, then that would be fine.
I honestly dont see why it is such an inconvenience to you for me to close the bugs that are marked resolved. You read thru every one of my emails because you dont trust my decisions, understandable, but if you are so worried about my marking a bug wrongly, then maybe you should stop developing, and strictly focus on helping clean up bugzilla, or the inverse could be true, maybe you could just trust that if I make a status change, it is the correct thing to do, and if not, then let someone else, who does have more time on their hands, catch it. Like you said, it's a small amount of bugs that are marked incorrectly. Of that percentage, I represent an even smaller amount, and like I said, I dont blindly close bugs, I read the majority of the comments, especially the ones toward the end, and the initial report, and then make a change only when I am confident that it is the right change to make.. -- Thanks Tom
On 5/31/07, Tom Spear <speeddymon(a)gmail.com> wrote:
On 5/31/07, James Hawkins <truiken(a)gmail.com> wrote:
I do read through all of your bug emails, which is exactly the problem, because I don't trust that you make the right decision on every bug, and in some cases I've had to go back and correct it. That is the issue.
It has been a small number of cases..
That small number forces me to read the rest.
The statement still stands. Resolved isn't resolved enough for you, but closed is...If a bug is marked resolved, I'm pretty sure it's resolved.
Obviously someone that commits patches to the upstream bugzilla tree agrees with me, because otherwise, there either wouldnt be a resolved option, OR there would be a close option on the bugs that are in any state of open, which is something I have suggested before, to cut down on list traffic.
We are our own project, and we use bugzilla differently, just like every other project out there has their own practices.
No one has problems with people correcting bugs that are marked incorrectly, but of the 30 or so bugs a day that you change, this case is a small amount. If you *only* changed this class of bugs, then that would be fine.
I honestly dont see why it is such an inconvenience to you for me to close the bugs that are marked resolved. You read thru every one of my emails because you dont trust my decisions, understandable, but if you are so worried about my marking a bug wrongly, then maybe you should stop developing, and strictly focus on helping clean up bugzilla,
huh? That doesn't make any sense. You're saying I should take even more time away from development to check your changes.
or the inverse could be true, maybe you could just trust that if I make a status change, it is the correct thing to do, and if not, then let someone else, who does have more time on their hands, catch it.
I'm not the only one that reads each change, and even if I were, and someone else took over, it's still wasted time on their part.
Like you said, it's a small amount of bugs that are marked incorrectly. Of that percentage, I represent an even smaller amount, and like I said, I dont blindly close bugs, I read the majority of the comments, especially the ones toward the end, and the initial report, and then make a change only when I am confident that it is the right change to make..
You continue to miss the point from the very beginning: you're trying to fix a problem that doesn't exist. There's nothing wrong with old bugs being left as resolved and marking new bugs closed on a case by case basis. -- James Hawkins
On 5/31/07, James Hawkins <truiken(a)gmail.com> wrote:
Obviously someone that commits patches to the upstream bugzilla tree agrees with me, because otherwise, there either wouldnt be a resolved option, OR there would be a close option on the bugs that are in any state of open, which is something I have suggested before, to cut down on list traffic.
We are our own project, and we use bugzilla differently, just like every other project out there has their own practices.
Ok.
I honestly dont see why it is such an inconvenience to you for me to close the bugs that are marked resolved. You read thru every one of my emails because you dont trust my decisions, understandable, but if you are so worried about my marking a bug wrongly, then maybe you should stop developing, and strictly focus on helping clean up bugzilla,
huh? That doesn't make any sense. You're saying I should take even more time away from development to check your changes.
No, I'm saying that if someone higher up in the heirarchy were to do this as well, then I wouldnt have as many bugs to mess with, and therefore I wouldnt be able to make as many bad changes.
I'm not the only one that reads each change, and even if I were, and someone else took over, it's still wasted time on their part.
I know you arent the only one who reads each change. What I'm saying is that someone who has more free time (someone who's time it wouldnt be wasting, because they can't do anything atm) should join the fun.
You continue to miss the point from the very beginning: you're trying to fix a problem that doesn't exist. There's nothing wrong with old bugs being left as resolved and marking new bugs closed on a case by case basis.
It's a problem to me, it may not be a problem to you, but that doesn't make it an invalid point. Marcus and Dan have both said to keep going, I'm sure others here (I'm not trying to speak for anyone, so someone else feel free to correct me if I am wrong) dont have any problem with my doing that either. And you miss the point as well, most new bugs that are marked resolved dont end up being closed on a case-by-case basis, which is why I am going back and doing that! Case in point: http://bugs.winehq.org/show_bug.cgi?id=7885 has been resolved for over a month, why was it not closed? http://bugs.winehq.org/show_bug.cgi?id=8373 has been resolved for 2 weeks, same question applies. -- Thanks Tom
On 5/31/07, Tom Spear <speeddymon(a)gmail.com> wrote:
It's a problem to me, it may not be a problem to you, but that doesn't make it an invalid point. Marcus and Dan have both said to keep going, I'm sure others here (I'm not trying to speak for anyone, so someone else feel free to correct me if I am wrong) dont have any problem with my doing that either.
And you miss the point as well, most new bugs that are marked resolved dont end up being closed on a case-by-case basis, which is why I am going back and doing that!
Case in point: http://bugs.winehq.org/show_bug.cgi?id=7885 has been resolved for over a month, why was it not closed? http://bugs.winehq.org/show_bug.cgi?id=8373 has been resolved for 2 weeks, same question applies.
Tom, This is how I finalize bugs: * When a bugs has decisively been fixed, by a merged patch, with test cases, or reported by user been fixed, then I close it. If it's decisively "not a wine bug" close invalid. * When a bug is rumored to be fixed, forgotten, or simply doesn't appear on your end, then probably resolve fixed, abandoned, or worksforme appropriately. * When the bug has been only set to resolved, and continues to have erroneous activity (i.e. commenting from random visitors that don't understand the report), then close it to discourage use of the bug. * If it's a bug that I don't know anything about, I shouldn't touch it. So the reason is, only "resolved" could maybe get revisited, and "closed" I never want to see again. Other than that, it makes no sense to me to close bugs unless there is some activity related to it (i.e. it is proved that an uncertain fix has really been fixed and the issue is done). Simply closing bugs worries me as does James. It really does need to be case-by-case, so what we know you are doing is right. Jesse
On 5/31/07, Jesse Allen <the3dfxdude(a)gmail.com> wrote:
Tom,
This is how I finalize bugs:
* When a bugs has decisively been fixed, by a merged patch, with test cases, or reported by user been fixed, then I close it. If it's decisively "not a wine bug" close invalid. * When a bug is rumored to be fixed, forgotten, or simply doesn't appear on your end, then probably resolve fixed, abandoned, or worksforme appropriately. * When the bug has been only set to resolved, and continues to have erroneous activity (i.e. commenting from random visitors that don't understand the report), then close it to discourage use of the bug. * If it's a bug that I don't know anything about, I shouldn't touch it.
So the reason is, only "resolved" could maybe get revisited, and "closed" I never want to see again. Other than that, it makes no sense to me to close bugs unless there is some activity related to it (i.e. it is proved that an uncertain fix has really been fixed and the issue is done). Simply closing bugs worries me as does James. It really does need to be case-by-case, so what we know you are doing is right.
Thanks for the clarification. So if it is already resolved as anything other than fixed, just leave it alone unless it continues to get activity. I still don't like it, but as with anything, majority rules, so I will stop. Sorry for the spam everyone. -- Thanks Tom
On 5/31/07, Tom Spear <speeddymon(a)gmail.com> wrote:
On 5/31/07, Jesse Allen <the3dfxdude(a)gmail.com> wrote:
Tom,
This is how I finalize bugs:
* When a bugs has decisively been fixed, by a merged patch, with test cases, or reported by user been fixed, then I close it. If it's decisively "not a wine bug" close invalid. * When a bug is rumored to be fixed, forgotten, or simply doesn't appear on your end, then probably resolve fixed, abandoned, or worksforme appropriately. * When the bug has been only set to resolved, and continues to have erroneous activity (i.e. commenting from random visitors that don't understand the report), then close it to discourage use of the bug. * If it's a bug that I don't know anything about, I shouldn't touch it.
So the reason is, only "resolved" could maybe get revisited, and "closed" I never want to see again. Other than that, it makes no sense to me to close bugs unless there is some activity related to it (i.e. it is proved that an uncertain fix has really been fixed and the issue is done). Simply closing bugs worries me as does James. It really does need to be case-by-case, so what we know you are doing is right.
Thanks for the clarification. So if it is already resolved as anything other than fixed, just leave it alone unless it continues to get activity. I still don't like it, but as with anything, majority rules, so I will stop. Sorry for the spam everyone.
-- Thanks
Tom
Remember that this is only my opinion. Other people handle things a little different. It is probably true that the two recent bugs you mentioned I would have closed, but it was only resolved. It's just how it was handled. If you want to close bugs that are clearly done with like that, it's fine by me. And I do close bug reports with other status too, if it's one I really don't want to see again. If it's clear why you are closing bugs, it's all fine by me, whatever it is. Jesse
On 5/31/07, Jesse Allen <the3dfxdude(a)gmail.com> wrote:
Remember that this is only my opinion. Other people handle things a little different. It is probably true that the two recent bugs you mentioned I would have closed, but it was only resolved. It's just how it was handled. If you want to close bugs that are clearly done with like that, it's fine by me. And I do close bug reports with other status too, if it's one I really don't want to see again.
If it's clear why you are closing bugs, it's all fine by me, whatever it is. Just to clarify my reasoning, because I just now thought about the _real_ reasoning lol, I often look thru lists of bugs that are resolved as fixed and (although i havent done it in a few years, i was going to start again), post a message to ask if the reporter can confirm if it is fixed. So I was closing bugs that were invalid/abandoned/dupe/worksforme so that they wouldnt show in the lists of resolved bugs, so its less I have to sort thru....
-- Thanks Tom
On 5/31/07, Tom Spear <speeddymon(a)gmail.com> wrote:
On 5/31/07, Jesse Allen <the3dfxdude(a)gmail.com> wrote:
Remember that this is only my opinion. Other people handle things a little different. It is probably true that the two recent bugs you mentioned I would have closed, but it was only resolved. It's just how it was handled. If you want to close bugs that are clearly done with like that, it's fine by me. And I do close bug reports with other status too, if it's one I really don't want to see again.
If it's clear why you are closing bugs, it's all fine by me, whatever it is. Just to clarify my reasoning, because I just now thought about the _real_ reasoning lol, I often look thru lists of bugs that are resolved as fixed and (although i havent done it in a few years, i was going to start again), post a message to ask if the reporter can confirm if it is fixed. So I was closing bugs that were invalid/abandoned/dupe/worksforme so that they wouldnt show in the lists of resolved bugs, so its less I have to sort thru....
Bug triage is a good idea so that stuff like this gets cleaned up. I'm not sure what everyone wants though. I guess you can figure that out :P
On 5/31/07, Jesse Allen <the3dfxdude(a)gmail.com> wrote:
Bug triage is a good idea so that stuff like this gets cleaned up. I'm not sure what everyone wants though. I guess you can figure that out :P
See, my opinion of what triage is isnt the same as everyone else's.. Either way, I'll just leave resolved bugs resolved, until they stop me from finding a specific bug I'm looking for. Allow me to get a consensus. Random Closing of Resolved bugs that have no activity is not deemed helpful by everyone, even if it does help me, so would it be acceptable for me to close a few (less than 20) a day? That way I'm not spamming the list for any more than 30 mins, and I can still kind of make it easier to find the bugs that need a followup. -- Thanks Tom
I'm not sure there is a agreement what some things here mean. The following is my understanding of things, please correct me or state differing understanding: triage bugs: Make sure the bug is properly filed, has enough information and possibly uncover the cause (e.g. regression testing, finding where a NULL that causes a crash inside the application comes from). This also includes marking a bug resolved,fixed or closed or whatever, but the prior thing is more important because it makes it easier to fix. resolved,fixed: I only mark bugs where I'm confident that they are really fixed as this. So if I need to ask the reporter or some user if it now works for them I do this before resolving it. I think I never "closed" a bug. To detect e.g. resolved bugs with new comments (e.g. requesting reopen) I run a query for changed bugs (where I made a comment) since last date up to which I queried this (I noted that down) and e.g. yesterday. Closing bugs doesn't help here either as they could be closed in error, so someone would still want to request those to be reopened. So is someone really using the "closed" status (not in the sense that they set it but e.g. use it in queries)? On Thu, May 31, 2007 at 05:36:45PM -0500, Tom Spear wrote:
So I was closing bugs that were invalid/abandoned/dupe/worksforme so that they wouldnt show in the lists of resolved bugs, so its less I have to sort thru....
Does closed convey any more meaning than resolved as invalid/abandoned/dupe/worksforme? I mean who would mark a bug as resolved if that is not the conclusion and not reopen when that was done in error? So isn't closing perhaps something we _really_ want to avoid doing too prematurely? Perhaps something we only do every major release ( like 0.9 ). Otherwise it looses it's meaning as "this is something we never ever need to look at". Jan
On 5/31/07, Jan Zerebecki <jan.wine(a)zerebecki.de> wrote:
I'm not sure there is a agreement what some things here mean. The following is my understanding of things, please correct me or state differing understanding:
triage bugs: Make sure the bug is properly filed, has enough information and possibly uncover the cause (e.g. regression testing, finding where a NULL that causes a crash inside the application comes from). This also includes marking a bug resolved,fixed or closed or whatever, but the prior thing is more important because it makes it easier to fix.
Agree
resolved,fixed: I only mark bugs where I'm confident that they are really fixed as this. So if I need to ask the reporter or some user if it now works for them I do this before resolving it. I think I never "closed" a bug.
Agree
To detect e.g. resolved bugs with new comments (e.g. requesting reopen) I run a query for changed bugs (where I made a comment) since last date up to which I queried this (I noted that down) and e.g. yesterday. Closing bugs doesn't help here either as they could be closed in error, so someone would still want to request those to be reopened.
True, bugs could be closed in error as well, which is why I don't close a bug, unless I was working with the reporter, or it is over (eg) 6 months since the last comment, and is already resolved.
So is someone really using the "closed" status (not in the sense that they set it but e.g. use it in queries)?
No, but that was my point. I search thru resolved bugs, to double check that users are satisfied with the result, and it does me no good to do that if I am searching thru bugs that are >6months old. If it is closed, a query for resolved bugs will not find those that are closed
On Thu, May 31, 2007 at 05:36:45PM -0500, Tom Spear wrote:
So I was closing bugs that were invalid/abandoned/dupe/worksforme so that they wouldnt show in the lists of resolved bugs, so its less I have to sort thru....
Does closed convey any more meaning than resolved as invalid/abandoned/dupe/worksforme? I mean who would mark a bug as resolved if that is not the conclusion and not reopen when that was done in error? So isn't closing perhaps something we _really_ want to avoid doing too prematurely? Perhaps something we only do every major release ( like 0.9 ). Otherwise it looses it's meaning as "this is something we never ever need to look at".
See above, however you are onto something with the only closing stale bugs at major releases.. According to James, the standard for marking a bug abandoned is 6 months from the request of more information without any response from the reporter or someone else having the same issue, and also not reproducible via a download. Perhaps we could do something similar to what Jonathan did before 0.9, say ping every bug at the 1.0 code freeze, and then resolve AND close any with no response, or that the reporter replied saying it is not an issue anymore.. With that in mind, once 1.0 goes live, will we still be doing monthly stable 1.0x releases, or will the release cycle be more of any x.0x releases are development and the stables will be x.x or x.5? However it is done, I think it would be a good idea to do pings of bugs prior to any STABLE release, and during development release periods, just ping when more than 6mos old like we currently are doing. -- Thanks Tom
participants (6)
-
Ben Hodgetts -
James Hawkins -
Jan Zerebecki -
Jesse Allen -
Marcus Meissner -
Tom Spear