Re: Help needed with Moodle 2.5 QA testing - my report, and questions

Re: Help needed with Moodle 2.5 QA testing - my report, and questions

by Séverin Terrier -
Number of replies: 9
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

Hi Helen,

During my QA tests, i've seen different things, like testing instructions that could be improved..., and hesitated several times to know what to do.

Some exemples :

I think that for some (or all?) of them, (related) issues should be created in the tracker, or someone will take care of that with existing information?

Don't hesitate to reset tests if needed, and/or answer my questions here and/or in the MDLQA issues.

Séverin

Average of ratings: Useful (1)
In reply to Séverin Terrier

Re: Help needed with Moodle 2.5 QA testing - my report, and questions

by Helen Foster -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Séverin,

Many thanks for highlighting all these issues related to QA tests. approve

Regarding testing instructions needing improving, you've been doing the right thing in adding me as a watcher and then adding a comment, as mentioned in the QA testing guidelines. I am gradually going through such issues and amending the testing instructions in the master copy of the test.

Regarding browser differences, I would recommend that if in doubt, you should mark the issue as failed, so that it can be investigated by a developer.

Please create a Moodle (MDL) issue for any suggested improvements to Moodle. For auto-linking MDLQA issues on moodle.org, please create a Moodle community sites (MDLSITES) issue.

Average of ratings: -
In reply to Helen Foster

Re: Help needed with Moodle 2.5 QA testing - my report, and questions

by Séverin Terrier -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

I was hoping that someone else (perhaps you Helen) could create issues (to save my time), having talked about it...

I've created MDLSITE-2233 for MDLQA issues auto-linking.

Will try to create the other ones.

Séverin

Average of ratings: -
In reply to Séverin Terrier

Re: Help needed with Moodle 2.5 QA testing - my report, and questions

by Helen Foster -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Thanks Séverin for creating MDLSITE-2233 and apologies for not having had chance to report issues myself.

For any QA tests which are marked as failed, if a tester doesn't have time to report the problem, then myself or someone from HQ will certainly do so.

Average of ratings: -
In reply to Helen Foster

Re: Help needed with Moodle 2.5 QA testing - my report, and questions

by Séverin Terrier -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

Thanks Helen.

I suppose that someone will create entries for real bugs, that can affect next Moodle version, but not sure take time to create related things, not affecting directly new version wink

Just to mention that in my first message, i've created manually the links to the MDLQA- entries. So, just to try, i write now MDLQA-5582 (with no manual link).

Séverin

Average of ratings: -
In reply to Séverin Terrier

Re: Help needed with Moodle 2.5 QA testing - my report, and questions

by Séverin Terrier -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

I also have some other remarks about QA testing, and who do what.

Is the general policy "the more eyes the better", and "anybody/the first to verify/validate marks the test as pass" (even if somebody else had allready begun working on that), or should we define a way to avoid several people spending time on testing the same tests?

I'm asking because when i've worked on QA testing, i've done this way :

  • if, opening a test (affected to "nobody"), i see that someone else as begun to work on this (letting a comment/question), i let it to him.
  • i've spend time to prepare all the pre-requisites to test the "Backup and restore activity with files reference" series, and then seen that somebody else had just made tests.
  • on MDLQA-5428, i've made tests, created bug report, said i'll test again, and somebody else passed the test (little frustrating)

Just to know how to best coordinate community efforts for better QA testing, and efficient work for better new versions.

And also, it could be useful to have on qa.moodle.net more existing materials to make the tests (example : MDLQA-5626 or MDLQA-5652), because sometimes it takes (a long) time to prepare all needed things, and you loose everything in less then 1 hour...

Séverin

Average of ratings: Useful (2)
In reply to Séverin Terrier

Re: Help needed with Moodle 2.5 QA testing - my report, and questions

by Jasmin Klindzic -
Picture of Testers Picture of Translators

I completely agree with all the things that Severin has said, both in this last post, and the original one.

Some of the test were detailed in describing all the steps and providing the necessary test materials, while others were missing a (crucial) step or two. :\

But in the retrospective overall experience is: Wow, it all went so fast (including the additional bugs and issue solving on behalf of developers in charge)! :D

 

Average of ratings: -
In reply to Séverin Terrier

Re: Help needed with Moodle 2.5 QA testing - my report, and questions

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Séverin and thank your for all your useful remarks about QA testing in Moodle.

Regarding the point in your latest post, how to not spend time working on a test when in ignorance that someone else is working on the same test, I suggest clicking on the Assign to me link.

It is my understanding that, once one tester has assigned a test to themselves, then no-one else will bother to work on it.  I have done it quite a few times in the previous (Moodle 2.4) QA testing.

Can Helen please confirm that that is the best policy?

Joseph

Average of ratings: -
In reply to Joseph Rézeau

Re: Help needed with Moodle 2.5 QA testing - my report, and questions

by Helen Foster -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Thanks everyone for your comments and feedback on the QA testing process. I have updated the section on running tests in the QA testing guide so that it now says:

Select a test from the list. If you wish, you can click the 'Assign to me' button, so that nobody else chooses the same test to run. (If you then find you are unable to run the test, you can assign the issue to 'Nobody'.)

Please note though, that when a failed test is reset (after a bug is fixed), the assignee is automatically set to 'Nobody', since although it is a great help for the original tester to re-run the test, we don't want to assume that they have the time to do so.

Regarding having more test materials (activities, groups, data etc) on the QA testing site, we hope to improve the site in time for the next QA testing cycle.

As mentioned previously, so far over a hundred QA tests have been automated as part of the Behat project in Moodle and are now run automatically every week. The plan is to automate as many of the straightforward QA tests as possible and remove them from the testing cycle, then add more exploratory QA tests covering new features and improvements in the upcoming version.

Having the help if the Moodle community in QA testing is wonderful and much appreciated. smile

Average of ratings: Useful (1)