Random questions (not) repeated across quizzes

Random questions (not) repeated across quizzes

by Pete Jones -
Number of replies: 8

Hi

This is a bit complicated to explain but here goes...

I'd like to have 4 separate quizzes in a course each selecting a question randomly from a bank of 4 questions. Each of the four questions is an embedded answers (cloze) question type with 60 multiple choice questions and marked out of 60.

The idea is that students will take the quizzes at regular intervals (10 weeks) and be able to track how their scores change, e.g. Quiz 1: 34/60, Quiz 2: 40/60. The quizzes have been benchmarked against each other so that they are a similar level of difficulty.

The reason why I'd like each quiz to select one of the four questions randomly is so that students taking the test at the same time/location may not get the same question.

The reason why I can't set it up as a single quiz with multiple attempts allowed (with a specified gap between attempts) is that the gradebook can only be set to show the first or last score, the average score or the highest score, but I want the gradebook to show the scores for each attempt.

I've discovered that having multiple quizzes set to pick a random question doesn't prevent a student getting a question they have had in a previous quiz.

Is there any way to either i) prevent a random question appearing twice for the same student when taking multiple quizzes that select from the same bank of questions, or ii) have the gradebook show the score for each attempt for a single quiz where multiple attempts are allowed?

I'm using Moodle 3.4.

Any help or suggestions will be appreciated.


Average of ratings: -
In reply to Pete Jones

Re: Random questions (not) repeated across quizzes

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

I am not an expert on this, but I will try to help.  (Tim might jump in at some point and clarify.)  I do regularly use randomized quizzes and multiple attempts.

Yes, the grade book will only show one grade for any single quiz, and you select the criteria.  If a quiz allows multiple attempts containing randomly selected questions, in the current version of Moodle, each attempt will select different questions until all questions have been selected once, and then the selection is randomized.

If you create two identical quizzes, yes, there is no way to prevent one quiz from picking a question that the other quiz has already picked.

A student who desires to see all the results of their multiple attempts can simply visit the quiz, and see each attempts score.  Likewise, the instructor can do the same.  Additionally, the instructor can visit the complete report for the student to see all their attempts and scores.

I think that the grade book is meant to show final scores on any grade book item.  If you want to use it to show intermediate scores and final scores, you would have to develop some specialized rules (calculations), which could be done, but would probably overwhelm most.

So I don't know of any way to make one quiz know which questions have already been picked in another quiz.

Average of ratings: Useful (1)
In reply to Rick Jerz

Re: Random questions (not) repeated across quizzes

by Pete Jones -

Thanks for confirming that, Rick

It's good to know that a student (and their teacher) can see all the results of their multiple attempts through a quiz itself (rather than the gradebook). I'll set up a single quiz and see whether this will work for our context.

I'll also have a look at Quiz Reports on Moodle docs as I'm not too familiar with this feature.

I'd be interested to know if you think there would be community interest in work to make one quiz know which questions have already been picked in another quiz. The purpose I need it for - tracking student progress through tests of comparable difficulty - would certainly be of use in other settings.

Pete



Average of ratings: -
In reply to Pete Jones

Re: Random questions (not) repeated across quizzes

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

Hi Pete,

Yes, get familiar with the quiz features that are provided by Moodle.  I work with the latest versions of Moodle (3.6.2+), and quiz features have improved through the year.

I don't know about the interest in making quizzes know what has occurred in other quizzes.  If I give a quiz on Chapter 2, why should this quiz need to know what the student has answered in Chapter 1?  Moodle's current method of allowing a quiz to be repeated has satisfied me for 10 years.  It allows students to repeat their learning and practice.

If one has a topic with different levels of difficulty, for example, 3, I would create three questions banks, and separate questions into each bank, then provide the students the ability to be quizzed at different levels, meaning three quizzes.  I might even intentional put some level 1 questions into the level 3 quiz, just to cause students to relearn a few things.

Moodle's quiz engine, and its question (database) bank are far superior to what other LMSs offer, but it does take a little effort to learn.

Having said all of this, I am not trying to discourage you from making a feature request on Moodle's Tracker system for your idea.  Tracker does not prevent ideas.  If you do so, the clearer that you can be about what you seek, and its advantages, will help your idea gain attention and votes.

Average of ratings: -
In reply to Pete Jones

Re: Random questions (not) repeated across quizzes

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

This is not something that could be developed as a neat plugin, because the relevant part of the quiz code does not have the necessary hooks in place to let plugin influence which questions are picked when a student starts a quiz attempt.

However, the relevant code is quite well structured, so you probably would not have to edit many line of code to change this. The key bits are

The rather cryptically named $qubaids object is the thing that represent which previous quiz attempts should be considered when picking random questions and trying to avoid duplicates.

Of course, hacking the code to change the behaviour is one thing. It would be a bit more work to add the options to put the teacher in control of this.

Average of ratings: -
In reply to Tim Hunt

Re: Random questions (not) repeated across quizzes

by Pete Jones -

Thanks Rick and Tim,

Rick, re "If I give a quiz on Chapter 2, why should this quiz need to know what the student has answered in Chapter 1", the use I'm thinking of is if you want to use a quiz to test someone on the entire course by giving them randomly selected tests of similar difficulty at different intervals throughout the course.

This is how I'm planning to use it, with the offline course being from beginner to advanced in a second language - it's part of our assessment of student progress with their language proficiency.

A similar use might be to use quizzes that test students on their readiness to take an exam (the new MEC perhaps) so they can see their progress at regular intervals even though they haven't completed the course. It could be particularly useful when whole tests or groups of questions are benchmarked against each other for difficulty level.

Having said all that, I now think that a single quiz will work for our context because students will be able see their results from each attempt in the quiz itself rather than the gradebook, so thank you again for letting me know about that, and thank you Tim for taking the time to look at this from a plugin/coding perspective.

Average of ratings: -
In reply to Pete Jones

Re: Random questions (not) repeated across quizzes

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I single embedded answers (cloze) question containing 60 sub-questions strikes me as a really bad idea. In theory it should work, but I really wount not want to rely on that. Also, the user-experience is going to be pretty bad for students.

To be honest, I woudl just put all 240 multiple-choice questions in a category, and let the quiz pick randomly. So what if you get, on average, 15 dupilcate questions between any two quizzes? Are your students likely to remember in that much detail which questions they have already seen a couple of months before?


Average of ratings: -
In reply to Tim Hunt

Re: Random questions (not) repeated across quizzes

by Pete Jones -

Thanks for your reply, Tim

I can't separate the questions unfortunately as they are testing the use of language items (e.g. vocabulary, grammar forms) in authentic texts, each texts is more difficult than the last, and the whole test of 60 questions is benchmarked against other tests to ensure a similar level of difficulty.

We've been using a similar quiz for a couple of years with reliable results - see screenshot below - and on the whole the user experience is good.

The only issue is that the timer disappears when the students scroll down the page.


Average of ratings: -
In reply to Pete Jones

Re: Random questions (not) repeated across quizzes

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I am pleased to learn that it has been reliable. Obviously, we try quite hard to make the Moodle quiz completely reliable, and on the whole it is, but what happens in extreme cases is always a worry.

Average of ratings: -