Part of improving the quality of Moodle is also improving the quality of tests. (open assignments is another subject.)
- When I saw the first draft of QTI, I was thrilled: just draw in your favorite elo-tool a map, do some QTI-scripting and then ask the user to click with the mouse on the city they think it is the Capitol of...
And then export this quiz to another system and still they can point with the mouse! But then realizing how much work it was to get it in the standard-QTI-format and asking your self how much better this "pointing with the mouse" was then just a map (with marks a,b,c and d) and asking in a M.C: "what is the Capitol.. a,b,c or d).
On that moment I was healed from QTI for the next two years at least.
Of course, when the whole world delivers test-sets in QTI-format it becomes interesting to have a QTI-importfilter, Until that day a simple structure format is my wish. - Another point is that teachers are asking for other things then these boring m.c.'s..
"Why not play the Hotpotatoes-blues inside Moodle?" Students want to taste variation in their brain-meals, is then the motivation.
I accept this for the phase where teachers want to invite students to work with the resourses and use tests for monitoring these activities, but not for grading. (So yes please create lots of variations like the missing word, or matching or.. and yes make the design as simple as possible: the missing word is brillant in it's simplicity.) - Coming to grading: How good is the test YOU just created as grading-tool: How fair are your weights? Are the (Wrong) alternatives in a question not too misleading?
How to callibrate all this, ending up with a test with realibilty or even predicting power.... - Everything you can describe as a set of m.c.-questions (like the missing word) can you even with "common sense" evaluate against the scores of your students, but how to check the quality of more complicated tests like a matching(words)-test?
For me the test problem has these parts:
- Creating m.c.-questions the most easy way AND question by question online AND with file in like this nice Missing-word-format
- Then organise in a database-table with a webfront the grouping and weights of questions in tests. (I think this interface-part is very nice done in PHPtest: http://www.resynthesize.com/code/phptest_info.php also the possibilty to give students debrief-information inside each question )
- Next point is the handling of the tests inside your Moodle (AND MAYBE ALSO in a kind of course-independent test-center)
- If you collect all the results of your students the next years then having a tool for question-analysis and test-callibrating against your own local population would be nice: must this become part of Moodle? Well, a simple histogram for the cumulative answers over the years for each each question would be nice, the same on test-level: the results of a test against other gradings inside a student cohort.
- ... But calibrating a test against studentsucces inside your local schoolsystem? I have no idea