Random Quiz Questions Take Forever to Grade

Random Quiz Questions Take Forever to Grade

by Juan F -
Number of replies: 14

Good afternoon folks!  I'm using Moodle 3.4.  I teach a course with 20 students.  I have created a weekly quiz that includes 10 open-response questions for the course.  I have a test bank with about 25 different questions and the quiz is randomized.  When I go to grade the quiz after it closes, I end up having to grade TONS of individualized questions...oftentimes, it takes me a few hours.  I'm pulling my hair out!  Please help.


In reality, I had hoped that Moodle would lump the same questions together, regardless of if they are randomized or not.  My goal of randomizing them was to limit cheating...but in the end, I'm killing myself.


Any suggestions?

Average of ratings: -
In reply to Juan F

Re: Random Quiz Questions Take Forever to Grade

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

If you are grading manually, I am not sure what lumping them together would gain you.  Aren't all the student answers different anyway?  Lumping the questions together is not going to reduce the number of answers...

Average of ratings: -
In reply to Emma Richardson

Re: Random Quiz Questions Take Forever to Grade

by Juan F -

Hi Emma, I see what you are saying about having to grade manually...however, opening 200 different questions is definitely NOT easier than grading 20 of the same question on a single screen/page.  Hope this makes sense.

Average of ratings: -
In reply to Juan F

Re: Random Quiz Questions Take Forever to Grade

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

Juan, have you tried clicking on the quiz, and from the "gear" dropdown, pick "Manual Grading"?  I think this should show all of the questions, individually, as you wish.

Average of ratings: Useful (1)
In reply to Rick Jerz

Re: Random Quiz Questions Take Forever to Grade

by Juan F -

Hi Rick, this is when I see over a hundred different questions...they all show 1, 2, 3, 4, or maybe 5 of the same question from different students but that's rare.  The algorithm to randomize works great...lol....it's the grading that takes forever.  I have to click each question and they show up multiple times.  I'm assuming it orders by when it appears in the quiz...thus, now lumping them together.

Average of ratings: -
In reply to Juan F

Re: Random Quiz Questions Take Forever to Grade

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

Juan, I will attach a screenshot of what I see for the one time that I used this technique.    In my case, I had different versions of questions 2, 3, 4, 5, etc.  When I go into manual grading, I can see the question number, and then the Question Name reflects the version of that question.

In your case, I am wondering if, in your quiz, every question from your question bank was used, so you see them all.  But even if so, might you still be able to grade each individual question, in batch.

If you look at my screenshot, I was able to grade "Q#2 Random sample of 20"  all at the same time, and there were 7 students who got this question.  10 students got the next question for this set, and 12 students got then next, etc.

Attachment Random.jpg
Average of ratings: Useful (1)
In reply to Rick Jerz

Re: Random Quiz Questions Take Forever to Grade

by Juan F -

Rick, this is close to mine but I have a question bank with 21 questions.  The quiz has 3 questions.  The class usually has 15-18 students.  See the screenshot below of what mine looks like....the list goes on to show Q #3 and has about 10 more of these.  Keep in mind, this is just one quiz in the course with 3 questions.  I have many others and on the exams, there are about 10-15 open-response/short-answer questions...



Average of ratings: -
In reply to Juan F

Re: Random Quiz Questions Take Forever to Grade

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

Okay, Juan.  But I must still be missing something.

Yes, your quiz delivered all 21 questions to students.  It appears that you have 11 versions of essay question #1.  Couldn't question #1, version 3 (for example) have a completely different answer than question #1, version 4?  So wouldn't you need to look at these individually?

I think what you are saying is that you want to grade all question #1 as a group, not individually.  And I think that is what Tim is pointing us to.  I think you are saying you would like to see all responses to question #1 at the same time.

Let's see if I can mock this up.

You might have a series of questions starting with: "How do you get from New York City to ..." and then have 10 versions, ending up with various cities like "Chicago," Los Angeles,"  "Houston", "Minneapolis", etc.  You might be looking for the common answer "by driving a car."  So by seeing all of these versions at one time, you could quickly see if "by driving a car" is part of your expected answer.

But the possible student's answer (Chicago) might not have anything to do with "driving a car."  For example, they might say take I80 west to xyz, turn north, go 50 miles to abc, switch to Amtrak, get off in efg, fly to def.  Yet, this could be a correct answer.  So one might have to look at all answers to this specific question (Chicago) in order to grade them.

So maybe a lot will depend upon how one has made the versions of the questions.  But I think that I see what you are after.

This discussion reminds me of why I try to avoid essay questions. smile

Average of ratings: -
In reply to Rick Jerz

Re: Random Quiz Questions Take Forever to Grade

by Juan F -

Rick, you said it all much more gracefully than I did...I think my frustration got the best of me.  And yes, I try to avoid open-ended questions but this is how I can tell my students really understand something...and that they aren't just guessing a letter.  Or at least that's what I tell myself!  smile  Thank you for your time.

Average of ratings: -
In reply to Juan F

Re: Random Quiz Questions Take Forever to Grade

by Germán Valero -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Have you seen the awesome Essay (auto-grade) question type by Gordon Bateson?

The essay (autograde) question type allows an essay question response to be given a preliminary grade that is generated automatically based on one or more of the following characteristics of the response.

  • the number of words in the response
  • the number of characters in the response
  • the presence of one or more target phrases in the response

The automatic grade can be overridden by the teacher later.

Additionally, the teacher can set up grading bands that offer a non-linear grading scheme. In such a scheme, the grade awarded is that of the grading band in which the word/character count falls.



Average of ratings: -
In reply to Germán Valero

Re: Random Quiz Questions Take Forever to Grade

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

German, yes, a possible solution.

But somehow, I thought the concept of having someone write something (i.e., an essay) is that someone (the person grading) would actually read what was written.

This plugin doesn't appear to read what was written.

Or said a little differently, "plugin appear read to written was This what doesn't."

Average of ratings: -
In reply to Rick Jerz

Re: Random Quiz Questions Take Forever to Grade

by Germán Valero -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Gordon Bateson's Essay (auto-grade) plugin allows the teacher to set some pre-defined target phrases. These target phrases are the 'important concepts' that the student should include in the essay.


I think that this plugin is best suited for formative assessments, where the teacher wants the student to really write a real essay, but the computer can quickly check whether the student has indeed written all the important concepts that the teacher expected the student to include.

With this plugin, the student can re-submit an essay that the computer marked (scored)  as incomplete, until the essay has all the target phrases. Then it would be a good time for the human teacher to read the essay and approve the mark or highlight the grammar or conceptual errors in the essay.


The teacher can choose whether to show the target phrases that the student has met. That is very directed feedback.

Or the student can get a sub-optimal mark for the submitted essay, and the student must then think what important element the teacher expected to be included in the essay, that the student has not included yet. This would be my preferred style to use for University level essays, as too many of my students have little inclination to think carefully before answering a computer quiz (the result of too many multiple choice questions in many computer quizzes).


You can read my pathology question example with six target phrases in the plugin documentation. A few years ago, I used to make essay questions like this for paper-based (drill or for-real) exams, and the scoring guide had all the six target ideas that the student was expected to write, and each one was worth one point. Manually marking such  assessments for 50 students every week was not much fun, but I can now have the computer check for these six important ideas automatically. Although I ought to check all the essays and substract points for grammar, spelling, wrong answers or lies in the essay.


I have not yet used this plugin for summative assessments, but I will write the results when I have them.

Average of ratings: -
In reply to Juan F

Re: Random Quiz Questions Take Forever to Grade

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

Somehow I seem to think that I once did this.  Maybe I went into Quiz, Statistics, or something like that.  I only did this one, in one course, everything of course now has a grade, so I don't have the best way of remembering what I did.

Somehow, I remember thinking "I didn't know that I could do this?"

Let's see if others have any ideas.

Yes, I know what you mean.  I too was picking randomized essay questions, and it was faster to grade the exact question all at one time, then the next question, etc.

Average of ratings: Useful (1)
In reply to Juan F

Re: Random Quiz Questions Take Forever to Grade

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

This is an acknowledged problem (MDL-49626) that really ought to get fixed one day.

Average of ratings: Useful (3)
In reply to Tim Hunt

Re: Random Quiz Questions Take Forever to Grade

by Juan F -

That's the one...and I just voted on it.  Thank you, TIm.

Average of ratings: -