questions about helping with QA

questions about helping with QA

by Brian Jorgensen -
Number of replies: 2
Hey, All:


I would like to start helping with QA and code review, but have to admit to being a little confused with the process. Here are my questions:

  1. Is there a Moodle instance set up that automatically follows the MOODLE_19_STABLE branch (via daily or hourly updates) that everyone uses for the clicking part of the weekly code review?
  2. Is there a historical reason it is called "STABLE" if it has features being added to it in addition to bug fixes?
  3. Can anyone forsee obstacles if I choose to try to do some of this work every day, instead of only every Tuesday?
  4. Is there a formal or informal hierarchy, whereby someone new like me is asked to review less critical types of bugs? (Seems like a good idea for everyone!)
  5. Finally, how does everyone know what each other is testing in order to avoid duplication of effort? In other words, how are JIRA issues "claimed" for QA?

Thanks,

Brian
Average of ratings: -
In reply to Brian Jorgensen

Re: questions about helping with QA

by Helen Foster -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators
Brian, thanks a lot for offering to help with QA. approve

To do so, you need to checkout the latest MOODLE_19_STABLE and/or HEAD (preferably both). All the MOODLE_XX_WEEKLY tags are then generated automatically at 1:00 GMT on Wednesday morning.

Tuesday is the day on which we focus on testing, but you're welcome to choose issues from the QA pending 1.9.x list and test them on other days of the week.

As far as I know, there's no hierarchy regarding reviewing critical bugs. (Testers, if I'm wrong on this, please jump in and correct me. wink) If you're unsure about a particular bug, you can always test it and add a comment, then leave it for someone else to close.

Regarding claiming issues for QA, please see Development:Weekly Code Review for details of the process. (I've added you to the testers group in tracker, so you'll be able to edit issues.)

Thanks again for offering to help with QA. The more QA we can do, the better Moodle becomes!

(PS I'm going to leave the question about why it's called "STABLE" for someone else to answer. wink)
In reply to Brian Jorgensen

Re: questions about helping with QA

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Thank you for offering to help with testing.

The processes are fairly informal. That is, we are more interested in outcomes than how we get them. So I think that the most important thing is to communicate. That is, use your common sense to decide what to do, but make sure it is clear (from the comments you add to bugs, or forum posts) what you are doing,
so if there is a problem with it, someone can suggest a better way.


1. There are servers test.moodle.org and demo.moodle.org that can be used for testing, but I am not sure how or when they are updated.

I think it would be better if you can get your own Moodle installation. It is quite easy to install on any computer, and then you have complete control over what you are testing.

Ideally, get a version from CVS (see CVS for administrators) so it is easy to keep up-to-date.

For bonus marks, actually have at least 3 different Moodle installs. Moodle 1.9 is most important at this time, but copies of Moodle 1.8 and 2.0 that you can test are good to have. (The trick here is to set each one up with a different database prefix.)

And for extra bonus marks, at some point learn How to apply a patch so you can test things before they are checked in. (But that can really wait for later.)


2. Well, STABLE is a relative term. It is called stable because it is stable enough to run a live system off. And sometimes there is an argument about whether something is a missing feature or a bug.


3. Not at all. The more testing the better. A bug can be tested any time after it is marked fixed, and the sooner any problems with the fix are found, the easier it is to deal with them.

Testing day is only really there to make the developers do a bit of testing and code review, rather than just write code all the time.


4. No. Another way of looking at it is that the more critical the bug, the more testing the fix needs. As Helen suggests, if, after testing, you are unsure, just comment on the testing you have done, and let someone else judge whether that is enough to close the bug. Back to communicating really.


5. Certainly learn your way around Jira. The most useful feature for this sort of thing is that you can save any search you have set up, to be used again later. There are several pre-defined ones you can use (see http://tracker.moodle.org/secure/ManageFilters.jspa?filterView=popular), and the 'QA pending 1.9.x' one is probably the place to start. However, I have saved my own customised version of that which restricts to bugs fixed in the last couple of weeks, and changes the sort order. (I think that if a fix has been out there for more that a couple of weeks, and no one has complained about regressions, the fix is probably OK, and there are always enough recent bugs to test.)

To answer your specific question, each bug has a qaassignee field, and you can add yourself there to indicate you have started testing. (You may wish to customise you Jira view, so you can see that field in bug lists, I don't think it is there by default.)


Another suggestion is to focus on testing just a few areas of Moodle to start with, so you can become familiar with how those parts work.
Average of ratings: Useful (1)