Useful tips for helping with Moodle 3.3 QA testing

Useful tips for helping with Moodle 3.3 QA testing

by Tim Hunt -
Number of replies: 3
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I just wanted to offer some advice to anyone doing Moodle 3.2 testing. When you test things, the goal is to break anything that can be broken. Using names like "Test course" is not very likely to break anything,

What is much more likely to cause havoc is creating your test course (or whatever) with a name like "x < 1 && x > 0" since bits of that can easily get confused with HTML syntax, if the code is wrong.

Also, given the nature of PHP the strings "0" (a single digit 0) is also very likely to cause havok like MDL-50893, MDL-53964, MDL-16427, MDL-44981, MDL-20960, MDL-35620 (Wow! I did not realise how many times this had caused a problem. Note that list is not complete.)

In the past, ' (since quote) was a useful test, because of how things were stored in the database, but it is much less likely to cause issues now. Even so, 'Fergal.O'Brien@example.com' (or similar) is a good test email address. Also, other systems are not as good as Moodle at dealing with special characters. In other systems it can be very worthwhile testing with things like 'café' or '囲碁' but that is unlikely to be worth it in Moodle.

Does anyone else have good tips for making your testing more likely to find bugs?

Average of ratings: Useful (7)
In reply to Tim Hunt

Re: Useful tips for helping with Moodle 3.3 QA testing

by Renaat Debleu -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers

Dates and times can also be difficult to handle, I had a plugin test running on travis that was working fine - except between 17h and 18h in the afternoon, then all my tests failed. 

The reason was simple: 17h local time = 24h Australian time, so my tests did not like the Australian midnight hour.

From that moment, I regularly try as a date:

  • 5 minutes before&after a happy new year
  • 15 minutes before&after midnight in Sidney
  • 29 February

BTW, Gherkin Scenario Outlines are an excellent way in Behat to test several hours/dates - also the difficult ones.

In reply to Renaat Debleu

Re: Useful tips for helping with Moodle 3.3 QA testing

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Yes, and the days around when daylight saving starts/ends are also great times to find bugs.