Hi,
it's great to see that there is more useful documentation about Moodle tracker, and bug triage.
But i think triage should also be part to help fix easy bugs rapidly, so i've made MDLSITE-376.
A copy-paste of what's in my request :
When you create a bug, you can specify it's priority, well.
It would be good to have a field (or two) to be able to specify :
- if there is a solution provided (which helps to fix the bug)
- if the bug is easy to fix (typo, missing string...)
This would help seeing and closing simple bugs quickly, avoiding people to report them several times (which takes more time to find duplicates).
Hope this helps...
Séverin
Hi Séverin,
Thanks a lot for your suggested tracker improvements.
The tracker is a great tool, however the sheer volume of information it contains can become rather overwhelming. With around 25 new bug reports added each weekday, we really need to work out an efficient way of dealing with them, hence the bug triage documentation (still under construction, though comments always welcome
).
Regarding easy-to-fix bugs, currently they are often given a priority of "Trivial". Do you think this is sufficient, or are you suggesting an additional field?
Thanks a lot for your suggested tracker improvements.
The tracker is a great tool, however the sheer volume of information it contains can become rather overwhelming. With around 25 new bug reports added each weekday, we really need to work out an efficient way of dealing with them, hence the bug triage documentation (still under construction, though comments always welcome
).Regarding easy-to-fix bugs, currently they are often given a priority of "Trivial". Do you think this is sufficient, or are you suggesting an additional field?
My experience as a developer is that I am very good at recognising from the bug report whether it is something that is easy to fix, and I don't need any other field to tell me that.
I also notice that the only time anyone adds a "This must be really easy to fix" comment to a bug is when they really mean "This bug is irritating me a lot, and I want you to fix it ASAP". Almost always they don't know what they are talking about, the bug is not easy to fix, and I get really pissed off and shove the bug to the bottom of my list.
So triage to make the bug report really clear, with good steps to reproduce is wonderful, but someone else's opinion of how easy the bug is to fix is worthless. If you think it is so easy, attach a patch!
I also notice that the only time anyone adds a "This must be really easy to fix" comment to a bug is when they really mean "This bug is irritating me a lot, and I want you to fix it ASAP". Almost always they don't know what they are talking about, the bug is not easy to fix, and I get really pissed off and shove the bug to the bottom of my list.
So triage to make the bug report really clear, with good steps to reproduce is wonderful, but someone else's opinion of how easy the bug is to fix is worthless. If you think it is so easy, attach a patch!
Hi,
i must admit i'm new to using the Tracker (2-3 months), and don't really know how developpers look at it, to know wich bug working on. Would be happy to have answers to that
Helen talks about priority "Trivial" : reading it, i understand that it's not something that needs to be closed quickly. So, do developpers look at it, or don't care and look at it long time later ? Because if it's the case, it means that we sometimes keep long time really easy to fix bugs (like typos), that can be reported by several people, creating duplicates, and lost time for everybody
My main idea is just : fix easy bugs as soon as possible, to clear the tracker
I understand that most people can't know if bugs are easy to solve or not (but for typos), and may misuse this
Perhaps it would be better to let triage people use that, but you perhaps already use good filters that help you rapidly fix easy bugs...
Would be good to know how developpers use the tracker (priority...)
I'm not really interested in improving the tracker : just the way we use it (but it's perhaps perfect now), to improve Moodle, and help all the community to avoid losing time
Séverin
i must admit i'm new to using the Tracker (2-3 months), and don't really know how developpers look at it, to know wich bug working on. Would be happy to have answers to that
Helen talks about priority "Trivial" : reading it, i understand that it's not something that needs to be closed quickly. So, do developpers look at it, or don't care and look at it long time later ? Because if it's the case, it means that we sometimes keep long time really easy to fix bugs (like typos), that can be reported by several people, creating duplicates, and lost time for everybody
My main idea is just : fix easy bugs as soon as possible, to clear the tracker
I understand that most people can't know if bugs are easy to solve or not (but for typos
), and may misuse this Perhaps it would be better to let triage people use that, but you perhaps already use good filters that help you rapidly fix easy bugs...
Would be good to know how developpers use the tracker (priority...)
I'm not really interested in improving the tracker : just the way we use it (but it's perhaps perfect now), to improve Moodle, and help all the community to avoid losing time
Séverin
Well, I personally operate by customising the front page of the tracker to display. I've attached a screenshot.
Top left are the list of open bugs and tasks assigned to me, sorted by priority. That list is depressingly long at the moment, because I have been busy recently. As you can see, it is only the bugs and the tasks. My view is that in an ideal world, you should fix bugs before adding any new features - although the world is not ideal. Anyway at it's best that list has been down to 25 issues, about half of which are genuinely hard to fix, as they are because of flaws of the way the quiz code is currently structured, and the only way to fix them requires a fair bit of refactoring.
On the top left you have my morale boost - the score of bugs I have fixed. Underneath that I have a view of all open quiz and question issues assigned to other people, clicking on a name there take me to the corresponding search results. As you can deduce from there, as well as the 65 bugs in quiz, there are about 100 feature requests and suggestions for improvement assigned to me.
To set this sort of thing up for yourself, you need to click the configure: ON link. The thing on the left is a Show Saved Filter portlet, and the ones on the right are Filter Statistics portlets.
Of course, the other main way I interact with the tracker is via emails. I read all new bugs as they come in, and if I have the time, and problem is clearly described, and they are not too difficult to fix compared to how critical they are, they probably get fixed almost immediately.
But if they are not clearly described, or I am too busy, they go on the list to be dealt with later. When it comes to attacking the backlog, I both attempt the higher priority ones first, but I also look ahead down the list to try to pick out easier ones first.
Of course, I also get emailed whenever anyone changes a bug, so if someone triaging bugs adds a comment confirming that this bug can be reproduced in 1.9, and clears up the description so I know exactly how to reproduce it, then that might trigger me into action.
I suppose there is a third way I interact with the tracker, which is with my memory. It sometimes alarms me how much I can remember about what quiz bugs have been reported, and what I have fixed, so if something comes up in the forums, I can often remember enough to locate an existing bug report, or find the duplicate of a new bug.
To answer another question: my mind, priority should be entirely about how serious the bug is. How much inconvenience (or worse) it is causing for users. Which bugs to fix, in what order, is really a cost benefit analysis. The cost is how much time I, as a developer, think it will take me. The benefit is how much better it makes Moodle, and that can only be determined by the users, not me. That is what the Priority is there to record.
Top left are the list of open bugs and tasks assigned to me, sorted by priority. That list is depressingly long at the moment, because I have been busy recently. As you can see, it is only the bugs and the tasks. My view is that in an ideal world, you should fix bugs before adding any new features - although the world is not ideal. Anyway at it's best that list has been down to 25 issues, about half of which are genuinely hard to fix, as they are because of flaws of the way the quiz code is currently structured, and the only way to fix them requires a fair bit of refactoring.
On the top left you have my morale boost - the score of bugs I have fixed. Underneath that I have a view of all open quiz and question issues assigned to other people, clicking on a name there take me to the corresponding search results. As you can deduce from there, as well as the 65 bugs in quiz, there are about 100 feature requests and suggestions for improvement assigned to me.
To set this sort of thing up for yourself, you need to click the configure: ON link. The thing on the left is a Show Saved Filter portlet, and the ones on the right are Filter Statistics portlets.
Of course, the other main way I interact with the tracker is via emails. I read all new bugs as they come in, and if I have the time, and problem is clearly described, and they are not too difficult to fix compared to how critical they are, they probably get fixed almost immediately.
But if they are not clearly described, or I am too busy, they go on the list to be dealt with later. When it comes to attacking the backlog, I both attempt the higher priority ones first, but I also look ahead down the list to try to pick out easier ones first.
Of course, I also get emailed whenever anyone changes a bug, so if someone triaging bugs adds a comment confirming that this bug can be reproduced in 1.9, and clears up the description so I know exactly how to reproduce it, then that might trigger me into action.
I suppose there is a third way I interact with the tracker, which is with my memory. It sometimes alarms me how much I can remember about what quiz bugs have been reported, and what I have fixed, so if something comes up in the forums, I can often remember enough to locate an existing bug report, or find the duplicate of a new bug.
To answer another question: my mind, priority should be entirely about how serious the bug is. How much inconvenience (or worse) it is causing for users. Which bugs to fix, in what order, is really a cost benefit analysis. The cost is how much time I, as a developer, think it will take me. The benefit is how much better it makes Moodle, and that can only be determined by the users, not me. That is what the Priority is there to record.
Thanks a lot Tim for your long and detailed post
Like you, i really think we should begin with fixing bugs, before adding new features...
Glad to know that you look at all declared bugs, when they're declared, and so, have hability to close simple ones. And sometimes look at them also.
I still think a simple way to indicate easy bugs (typos...) would allow putting good filters on it, and speeding the resolution. Must we consider putting priority = "Trivial", and some comments is the good solution ?
Séverin
Like you, i really think we should begin with fixing bugs, before adding new features...
Glad to know that you look at all declared bugs, when they're declared, and so, have hability to close simple ones. And sometimes look at them also.
I still think a simple way to indicate easy bugs (typos...) would allow putting good filters on it, and speeding the resolution. Must we consider putting priority = "Trivial", and some comments is the good solution ?
Séverin
Well, look at the right hand column of http://tracker.moodle.org/browse/MDL. Most people have fewer than 50 bugs assigned to them, and the tracker displays 50 bugs per page in bug lists, so a list that long is relatively easy to keep on top of - that is assuming you have any time to work on Moodle - and of course, those are issues, not necessarily bugs.
The real problem is the 2000+ bugs that are assigned to Martin. This is because there are too many modules in Moodle without a designated maintainer. Therefore, too many categories of bug are assigned to Martin by default, and he cannot possibly look at them all. (I assume, perhaps he does manage it somehow.)
So, almost before we do more triage, we need to find more module maintainers, so there are people for the triage-ers to assign bug to to get them fixed.
However, that is unnecessarily negative. There is lots of useful triage that can be done.
But really, I don't think extra fields in the tracker are going to help much. If the scenario goes like this:
1. Someone files an easy-to-fix bug, say a typo.
2. The developer get emailed, but for whatever reason ignores the bug report.
3. A triage-er looks at the bug, confirms that it really is just a typo, and adds a comment to the bug saying this.
4. So the developer gets emailed again.
Well if the developer goes on ignoring the problem, then really, nothing, other than reassigning the bug to someone else, is going to get the problem fixed quickly.
Also, the tracker has perfectly good full-text search, so if you want to find all the quiz and question bank bugs that mention the word typo, then you can. As it happens, that only returns one hit, which is a false-positive, but it is perfectly easy to see that.
So, I don't see the need for a special field in the tracker to indicate easy to fix bugs. Just a clear description of the problem in the description and comments is sufficient.
When searching the key thing is to be able to get the search results down to about a dozen bugs or fewer. That is a short enough list to go through by eye to find the ones you really wanted. In fact, often when I have searched and found extra, 'unwanted' bugs, that has actually revealed duplicates. For example, searching on the name of a particular question type, like 'multichoice', is often a good way to find dupes. (Except that I have done that in the past, if you are going to play that game, try some other keywords.)
However, it would be really great to have a way to search for issues that have a patch attached. Actually, I have just tried, and searching for 'patch' in the comments (or '+patch +attach*', if you are feeling clever), seems to work well enough.
The real problem is the 2000+ bugs that are assigned to Martin. This is because there are too many modules in Moodle without a designated maintainer. Therefore, too many categories of bug are assigned to Martin by default, and he cannot possibly look at them all. (I assume, perhaps he does manage it somehow.)
So, almost before we do more triage, we need to find more module maintainers, so there are people for the triage-ers to assign bug to to get them fixed.
However, that is unnecessarily negative. There is lots of useful triage that can be done.
But really, I don't think extra fields in the tracker are going to help much. If the scenario goes like this:
1. Someone files an easy-to-fix bug, say a typo.
2. The developer get emailed, but for whatever reason ignores the bug report.
3. A triage-er looks at the bug, confirms that it really is just a typo, and adds a comment to the bug saying this.
4. So the developer gets emailed again.
Well if the developer goes on ignoring the problem, then really, nothing, other than reassigning the bug to someone else, is going to get the problem fixed quickly.
Also, the tracker has perfectly good full-text search, so if you want to find all the quiz and question bank bugs that mention the word typo, then you can. As it happens, that only returns one hit, which is a false-positive, but it is perfectly easy to see that.
So, I don't see the need for a special field in the tracker to indicate easy to fix bugs. Just a clear description of the problem in the description and comments is sufficient.
When searching the key thing is to be able to get the search results down to about a dozen bugs or fewer. That is a short enough list to go through by eye to find the ones you really wanted. In fact, often when I have searched and found extra, 'unwanted' bugs, that has actually revealed duplicates. For example, searching on the name of a particular question type, like 'multichoice', is often a good way to find dupes. (Except that I have done that in the past, if you are going to play that game, try some other keywords.)
However, it would be really great to have a way to search for issues that have a patch attached. Actually, I have just tried, and searching for 'patch' in the comments (or '+patch +attach*', if you are feeling clever), seems to work well enough.
Still thanks for all the usefull explanations
I was more thinking at the 2000+ bugs than the short list other developpers have
But if you tell me things are done to see (or search) simple bugs, and close them, i'm happy to read that.
I was more thinking at the 2000+ bugs than the short list other developpers have
But if you tell me things are done to see (or search) simple bugs, and close them, i'm happy to read that.
I think one avenue might be to have a field (or perhaps just a focused search) which indicates that a patch has been provided. Patches can get lost in the tracker just as much as 'easy bugs', particularly if attached to older bugs. It'd be great if we could at least try to ensure every single patch is reviewed.
I'm really keen to see efforts stepped up for bug triage! I did quite a bit of triage myself over the Christmas period and had a few thoughts on things we might be able to do (which I didn't write down). From memory I think there are a few problem areas we could work on:
1) Reclassifying new features marked as bugs
2) Bugs which we can't reproduce and don't get feedback from the reporter
3) Bugs which we can't reproduce and are a problem with local installation
4) Old bugs which aren't relevant anymore
In general, I think we should be more keen to close bugs, it helps developers focus and stops and endless list of bugs without a resolution. Unfortunately it comes across as negative and discounting the requesters opinion to the close a bug as 'wont fix/not a bug'. On the other hand, reviewing and deciding to close a bug is much better than leaving it open without a resolution for years IMO.
Why not triage through your oldest reported bugs today and see if they are still relevant?
I'm really keen to see efforts stepped up for bug triage! I did quite a bit of triage myself over the Christmas period and had a few thoughts on things we might be able to do (which I didn't write down). From memory I think there are a few problem areas we could work on:
1) Reclassifying new features marked as bugs
2) Bugs which we can't reproduce and don't get feedback from the reporter
3) Bugs which we can't reproduce and are a problem with local installation
4) Old bugs which aren't relevant anymore
In general, I think we should be more keen to close bugs, it helps developers focus and stops and endless list of bugs without a resolution. Unfortunately it comes across as negative and discounting the requesters opinion to the close a bug as 'wont fix/not a bug'. On the other hand, reviewing and deciding to close a bug is much better than leaving it open without a resolution for years IMO.
Why not triage through your oldest reported bugs today and see if they are still relevant?
I agree with Tim - the authority is the developer that is going to fix it
Excelent thread though. Some hard-nosed triaging of bugs will be excellent. Even a "steps-to-repro" team (that engages with the reporter until there is enough ste-by-step detail in the tracker to repro the bug reliably) as part of the QA effort can work wonders. This helps sort out the noise, and educates bug reporters in what kind of details we need to diagnose...
Excelent thread though. Some hard-nosed triaging of bugs will be excellent. Even a "steps-to-repro" team (that engages with the reporter until there is enough ste-by-step detail in the tracker to repro the bug reliably) as part of the QA effort can work wonders. This helps sort out the noise, and educates bug reporters in what kind of details we need to diagnose...
Triage: hopefully this will become an effective way to sort through the huge number of issues reported every day.
Additional field: see comments in MDLSITE-376. Reporters should include information in the 'description' field when they have insight into solutions for the reported problems.
Fantastic to see people are interested in improving an already great tool: Tracker.
Additional field: see comments in MDLSITE-376. Reporters should include information in the 'description' field when they have insight into solutions for the reported problems.
Fantastic to see people are interested in improving an already great tool: Tracker.
As an exemple, a little search just allows me to :