Martin D and I have been discussing the possibility of establishing an official Moodle QA group. This group would have responsibility for testing/verifying bugs as they are resolved by developers. I'm trying to gauge how many people would be interested in becoming members of this group.
Testers would have to know Moodle well or be interested in ramping up their skills. I image testers would specialise in specific areas of the product, but I know there are people with broad understanding.
Like Moodle development, testing would be cyclical. Testers would focus on bugs that are fixed for the next version to be released.
Thoughts? Ideas?
It seems to me bug fixers should to some extent also be testing their fixes as they would be able to pinpoint the remaining problems much faster than someone else doing the testing. If you hand that job over to a tester all they can identify for sure is the symptom, not the cause. The fixer could do both.
I think more important would be to have a team testing to identify unknown bugs in development versions of Moodle.
I think more important would be to have a team testing to identify unknown bugs in development versions of Moodle.
I think more important would be to have a team testing to identify unknown bugs in development versions of Moodle.
That is exactly what is being proposed !
The other useful contribution is triaging incoming bug reports, finding simple steps to reproduce, or testcases that clearly show the problem, which is of huge assistance to the developers.
Of corse the developers can do all this, but the more time they spend on other tasks, the less time they spend fixing the bugs, so joining a QA team would be an excellent way for non-PHP-programmers to help get bugs fixed.
Michael,
Feel tree to take over the quiz testing day I have planned. As long as lots of quiz testing gets done, I don't mind if other stuff happens too.
That is exactly what is being proposed !
The other useful contribution is triaging incoming bug reports, finding simple steps to reproduce, or testcases that clearly show the problem, which is of huge assistance to the developers.
Of corse the developers can do all this, but the more time they spend on other tasks, the less time they spend fixing the bugs, so joining a QA team would be an excellent way for non-PHP-programmers to help get bugs fixed.
Michael,
Feel tree to take over the quiz testing day I have planned. As long as lots of quiz testing gets done, I don't mind if other stuff happens too.
That's great news Helen - glad to know you're interested. I haven't heard from many others on the subject of a QA group. Any ideas on how we might be able to gererate more interest and enthusiasm? Perhaps floating the idea in a forum with a greater subscription base than the "Testing and QA" forum?
Nicole, any ideas?
Nicole, any ideas?
I think the problem is the regularity of the work: because of their "real" jobs, a lot of Open Source Software contributors are irregular, that is they work on it in "time slots" and then disappear for some time and reappear again, and so on (I do work so on OOS).
I you think that this way of work could do it for your test group, count me in! And you should probably state this to have more people in it.
(And please, forgive me for my "english"
)
I you think that this way of work could do it for your test group, count me in! And you should probably state this to have more people in it.
(And please, forgive me for my "english"
At this point there is no definitive
job description as ideas are still being mulling around.
In
many software development projects the team testing or verifying
individual changes act independently of developers. A second
set of eyes usually results in a higher level of confidence in the
fix. Ideally the testing team knows the product well and is a
regular contributor to Tracker.
Here's what I'd like to see:
A team of testers monitor Tracker for bugs that have been resolved
(status=resolved) for an upcoming release. Testers assign bugs
to themselves, and test them using the details in Tracker. If a
bug passes, the status is updated to "closed". If the
bug isn't fixed or has been implemented incorrectly (in the opinion
of the tester), the bug is "reopened", essentially sent
back to the developer for re-work. If, at the time of release,
every bug had been subjected to this process, we could be fairly
confident the overall quality of the release would be that much
higher.
Areas that need further consideration:
- Are there enough people to cover the testing effort? [This goes to the question of time commitment. Major releases with multiple feature updates will require a greater commitment than minor releases.]
- Do tester's skills adequately cover the changes
in any given release? For instance, how are testers "skilled up" for a major functional change such as Roles?
- Are we willing to live with a
partially tested release? (I'm sure the answer to this is yes, but it
would be great to see every bug independently tested.)
Tracker
is flexible and can be tailored to our needs. I'm guessing
there are enough people out there with an interest in further
strengthening Moodle. The benefits to the product are obvious,
but this could also be an opportunity for individuals to contribute
in new ways to the project.
I think for testers to do such a job well the developers would have to be a little more explicit in what they did to fix the bug, e.g. what files they changed.
I know sometimes I will come across a bug for which the developers have simply added a comment like "fixed in head" and I think boy I would like to implement that on my site but I don't even know what they changed to be able to do so.
It may not be necessary, but I still think for a tester to be able to give a developer more useful feedback the tester has to have some idea what is going on in the code itself.
I know sometimes I will come across a bug for which the developers have simply added a comment like "fixed in head" and I think boy I would like to implement that on my site but I don't even know what they changed to be able to do so.
It may not be necessary, but I still think for a tester to be able to give a developer more useful feedback the tester has to have some idea what is going on in the code itself.
1-I was talking more about a live site, not a test one. I've got a lot of hacks to my site and doing a whole site upgrade is a very painful process because some hacks have to be reworked when there is a version upgrade, so for little fixes, if I can just change a line of code here or there it is really helpful to know what has been changed.
But this raises the question, would testers be expected to have their own test sites or would test.moodle.com be made available and kept up to date?
2-I see no version control tab either.
But this raises the question, would testers be expected to have their own test sites or would test.moodle.com be made available and kept up to date?
2-I see no version control tab either.
1) We're talking about testing here but I recommend running live sites off CVS (all mine are). CVS is designed for managing local changes.
That said, if your site has a LOT of local hacks then it's unlikely your testing will be very useful, because the hacks could interfere with or mask "standard" behaviour.
It's a bit tricky keeping test.moodle.com updated daily unless we have testers maintaining it from the web side.
2) Ah, it was viewable by developers only. OK, try it again now.
That said, if your site has a LOT of local hacks then it's unlikely your testing will be very useful, because the hacks could interfere with or mask "standard" behaviour.
It's a bit tricky keeping test.moodle.com updated daily unless we have testers maintaining it from the web side.
2) Ah, it was viewable by developers only. OK, try it again now.
I would be happy to help with any testing. Our group currently runs two development versions that we test internally from CVS prior to moving to our production environment. One version has our hacks and the other is the clean CVS installation. Both environments run a backup copy of our production db with 500+ users and 30+ courses.
Let us know if there is anything we can do to help....
Skip
Let us know if there is anything we can do to help....
Skip
In reply to Martin Dougiamas
Vast: Re: Establish an official QA or testing group?
由Jani Lemmetyinen發表於
I would also like to help with any testing, but I'm not sure if I have enough time and/or skills. CVS is not familiar to me, and I have just learning to use Moodle Tracker. So far, I have tested mostly modules and blocks. Currenty we have one test site and one production site.
- Jani -
- Jani -
Count me in as well. I'd be glad to help out. We have a
couple development servers where we run both clean copies as well as
copies with nonstandard modules & blocks (book, questionnaire,
ipodcast, etc)
We use these systems to test Moodle releases before using them on our production server.
Let me know what I can do to help
-Steve
We use these systems to test Moodle releases before using them on our production server.
Let me know what I can do to help
-Steve
We have an opportunity to improve the quality of information in Tracker. I agree in many instances there is not enough info to reproduce bugs. The quality of the information in Tracker will improve as the testing team asks for/requires better and more detailed information about bugs. Release notes will be produced from Tracker, so the quality of these will improve too.
For what it's worth, I'll help. I'm going to start testing 1.7 next week and will be focusing on what it will take for a smooth upgrade of our Moodle installs from 1.6.1 to 1.7. I'll probably focus on those sorts of issues, so I don't know how 'official' I'll be, but I'll test and report what I find.
Thanks,
Matt
Thanks,
Matt
I got here via the front Community forum page block on latest news.
I can devote some time to testing as part of my job. I assist my site administrator with conversions and mentoring faculty. The first thing I do is to become familar with the next branch, without any custom hacks or even non-standard modules, in the areas we use the most (Lessons) and Quiz activities.
I'm still in my first development cyle going from 1.5.3 to 1.6.1. We will swich over sometime this month to 1.6.1 and then critique our process. I just started downloading the latest 1.7 every week. So I have 3 localhost versions going on a couple of machines.
My manufacturing background helps me relate but I have never managed software development, so I will listen to others talk about process. Seems we need a database or even a linked table with needs/areas and testers and timegoals.
I can devote some time to testing as part of my job. I assist my site administrator with conversions and mentoring faculty. The first thing I do is to become familar with the next branch, without any custom hacks or even non-standard modules, in the areas we use the most (Lessons) and Quiz activities.
I'm still in my first development cyle going from 1.5.3 to 1.6.1. We will swich over sometime this month to 1.6.1 and then critique our process. I just started downloading the latest 1.7 every week. So I have 3 localhost versions going on a couple of machines.
My manufacturing background helps me relate but I have never managed software development, so I will listen to others talk about process. Seems we need a database or even a linked table with needs/areas and testers and timegoals.
Josh - You're now an official member of the testing team! Please read http://docs.moodle.org/en/Tracker before you get started.
Comments/suggestions for improving the process are aways welcome.
Comments/suggestions for improving the process are aways welcome.
Great. We can use the help. You're now a member of the testing group.
Before you get started please read http://docs.moodle.org/en/Tracker for information on the defect lifecycle and how to correctly use Tracker.
Before you get started please read http://docs.moodle.org/en/Tracker for information on the defect lifecycle and how to correctly use Tracker.
Wecome to the Moodle testing group Teemu. Our first priorty is testing bugs resolved for Moodle 1.7. Please read http://docs.moodle.org/en/Tracker for info on how to use Tracker. Thanks for your help!
I am semi new to Moodle but I have had the privilege of diving head first into all things moodle for the last month as my company is looking to roll out a full function Moodle LMS for employee training. I would love to support the product as a tester. Send me the requirements you are looking for and I will try and exceed them.
Count me in!
Charles, you must create an account in Tracker (http://tracker.moodle.org) prior to becoming a tester. Please do this, familiarize yourself with Tracker, then drop me line - I'll then make the changes necessary. Don't forget to read the Tracker instruction document at http://docs.moodle.org/Tracker.