New inactive-issue-closing policy

New inactive-issue-closing policy

by Oleg Sychev -
Number of replies: 9
Picture of Core developers Picture of Plugin developers

Recently I (and probably not only I) got a number of issue tracker letters for old issues (bugs mainly) that they would be closed if no comment on their relevance would be done in a month. I.e. if no one in the Moodle HQ cares about a bug for the year, reporter expected to re-test them on new Moodle version or the bug would be closed. I'm not particulary sure this policy is best. Let's explain.

For me there are two kinds of bug reports: bugs that are particular nasty and really annoy me, and bugs that I just come across and spotted. I usually write bug reports even for the second kind of bugs as a way to give back and free investment to make Moodle better. Such policy makes me an author of quite a lot of bug reports.

Now if the bug is really important in the daily work I'd find time to re-test it on the new Moodle version in the month (we are still using 1.9 due to too many dubious "enhancements" in 2.x versions). But having limited time I may not find enought time and inclination to re-create a tests for rare bugs I just spotted in everyday work on some Moodle installation for the new versions (some bugs require quite an effort to reproduce). So they will be closed, but that doesn't really mean they are fixed. I may not be the only one doing so.

Now, the question for Moodle HQ - did you really want to forget about bug report if an original reporter doesn't have time to re-test it on the new Moodle version? You may lose a useful testing information. And for the bugs I care about, but Moodle HQ is not - would I be expected to re-test each bug every year in similar fashion until someone in Moodle at last look at it?

And question for the bug reporters - how do you feel about new policy and how much of you old bug reports you actually going to re-test?

Average of ratings: -
In reply to Oleg Sychev

Re: New inactive-issue-closing policy

by Dan Marsden -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators

Personally I think this is a great idea!

The quality of a lot of the old bugs is really poor - many are duplicated, many are incomplete, many are not actually real bugs and a lot have been fixed or are no longer relevant in newer versions.

ALL new bugs entered into the tracker are reviewed by a real person and put through a triage process - making sure it isn't duplicated, making sure it is a real bug, and making sure it is assigned to the correct maintainer.

Basically all those old bugs are "off the radar" anyway... Michael has just asked people to comment/confirm it's still really an issue - and if not we'll close it off.. all it takes is for someone to comment on those bugs and say "yes" it's still a problem in version X and HQ will then "triage" the old bug... putting it back on the radar - giving a better chance it will be looked at than previously.

I got a bunch of those e-mails myself and 50-60% of the issues I see in my list (haven't gone through them all yet) can be closed - others I will triage myself and add the correct versions etc to "keep it on the radar" - please please please! help us to triage the old bugs by telling the developers which ones are still relevant!!

I've already done this for all the SCORM bugs in the tracker - If I ask for more information and it isn't provided/no feedback is given within a month (sometimes I'm even more ruthless) - I close the bug to show that I'm not spending any more time on it. - If someone eventually provides more information to confirm the issue it's relatively easy (for the developer) to re-open the bug or for the person to create a new bug with all the required information and it will enter the triage process for new bugs.

Average of ratings: -
In reply to Dan Marsden

Re: New inactive-issue-closing policy

by Oleg Sychev -
Picture of Core developers Picture of Plugin developers

Well, the keywords are "still" and "version X" - for several of my bugs it take quite a work to recreate test situation on the newer versions and say "yes". The month is not easy and I doubt I could do them all till April or so. 

As for quality - you could tell that from report, not the comment. I always published steps to reproduce the bug if it possible. If anyone have trouble with the quality of my report, he could close if with "Couldn't reproduce".

Average of ratings: -
In reply to Oleg Sychev

Re: New inactive-issue-closing policy

by Hubert Chathi -

I'm sure that the one-month deadline is negotiable.  It doesn't hurt to add a comment to the issue saying something like: "I'm not sure if this issue is fixed, but I won't be able to test until [future date]."

It would have been nice, though, if HQ had given us advance notice that they were going to do this, along with rationale for why they were doing this.  It would have made people less angry.

Average of ratings: -
In reply to Hubert Chathi

Re: New inactive-issue-closing policy

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I agree.

 

Note that you can find these issues using queries like

comment ~ lqjjLKA0p6 AND (votes > 5 OR labels = triaged) ORDER BY votes DESC, issuetype ASC

in the tracker. The comment ~ lqjjLKA0p6 finds these issues. Incidentally, that specific query finds 45 issues that I think someone from Moodle HQ should review before they are auto-closed.

 

Also, from the Developer chat archive, here is how Michael announced it:

Michael de Raadt: I just gave notice for 1682 issues that appear to be neglected. If they are not commented on in the next month they will be closed. I think I just spammed a large number of people.

Michael de Raadt: For a reletive context we have 4394 unresolved Moodle bugs.

Average of ratings: Useful (1)
In reply to Dan Marsden

Re: New inactive-issue-closing policy

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I would not say that it is a great idea, but after initially being annoyed, I decided that actually it was a reasonably pragmatic approach to make efficient use of the limited developer resource.

Yes, we will lose a small amount of information, but not much, and it will save a lot of time.

Average of ratings: -
In reply to Oleg Sychev

Re: New inactive-issue-closing policy

by Michael de Raadt -

Thanks for your feedback on this, Oleg, Dan, Hubert and Tim.

There are numerous issues in the tracker that relate to problems that are no longer present. You are correct in your assumption that we at Moodle HQ do not re-test all issues after a release is made; this is not a viable task considering the scale of the issues in tracker. This is why we need to draw on the strength of the community to assist in keeping topical issues active.

If you believe that something may be an issue, even if you are not able to retest the issue, then please post a comment on the issue.

You may be interested in the criteria used to determine which issues were determined as neglected; they were as follows.

  • Reported as affecting only currently unsupported versions
  • Not a security issue
  • Untouched for over one year

It's difficult for us to know if something reported over a year ago for an older version still affects currently supported versions, so we are referring this back to the people involved in the issue. Many pre-2.0 issues are no longer relevant to current versions because of the changes that came about with 2.0, so we feel justified in undertaking this purge now (approximately one year after 2.0 release). We have no specific plan or policy to repeat this action again annually, but we'll see how this one goes.

Michael d;

Average of ratings: -
In reply to Michael de Raadt

Re: New inactive-issue-closing policy

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I would add some more criteria:

4. Is a bug. I don't think feature quests go stale in the same way. There are some very old quiz/question, but valuable feature requests that I had to go in and save.

5. Has fewer than X votes, where X might be about 5. Anything that many people care about should probably be re-tested manually, and either closed or properly triaged. Of course, you could argue that with that many votes, someone is bound to respond to the comment you added asking someone to comment if they don't want the issue to be closed.

6. Has not already been triaged. If someone has, at some point, read the issue, concluded that it is valid, and triaged it, do we really want to auto-close it?

Average of ratings: Useful (2)
In reply to Michael de Raadt

Re: New inactive-issue-closing policy

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Just to note that the axe has now fallen on ~1300 issues: http://tracker.moodle.org/browse/MDL

As a check, I decided to review all 80+ issues that I got emailed about, and I found 6 I thought were worth saving, so less than 1 in 12. 

Since the initial warning about the mass closing, serveral people looked at the bugs they were interested (I think in the initial mail-bomb I got nearer 130 emails) and many of the 50 that were included in that, but not the final 80 were manually closed after re-testing, but probably more than 6 of those were saved by people who cared about them commenting.

So, overall, I think the process was quite robust. Anything that is really still an issue is likely to be re-discovered and reported again in the tracker.

Average of ratings: -
In reply to Tim Hunt

Re: New inactive-issue-closing policy

by Nadav Kavalerchik -
Picture of Core developers Picture of Plugin developers Picture of Testers Picture of Translators

"Anything that is really still an issue is likely to be re-discovered and reported again in the tracker."

Indeed smile

Average of ratings: -