Responses to a single question missing from quiz results for all students

Responses to a single question missing from quiz results for all students

par Brian Warling,
Nombre de réponses : 15

Last week, we held a quiz with 65 questions, for 158 students. There are responses from all the students for all questions, except one. For one question, there are no responses from any of the 158 students. We know that the students answered this question. The instructor says that she did not change the question in the exam, or regrade it.

The questions were shown in order (as they appear on the edit screen), with a new page after every 6 questions. The problem occurs with the 58th question. This question is multiple choice with a single correct answer. There isn't anything unique or different about this question (from what we can tell). It looks like all the others (most of the questions are multiple choice).

I've attached a snippet of the quiz results page that shows the question with no responses, along with the response history for this question from one of the submissions (they all look like this) -- you can see that the response history says the question was not saved.

Has anyone seen this before and might know what is happening? Is there a way we can get the responses back for this question?   And is there a way we can prevent this in the future?

We are running Moodle 2.4.6 (Build: 20130909). This quiz was originally on our Moodle 1.9 server and was moved to Moodle 2.4.

Thanks much.... Brian

 

Moyenne des évaluations  -
En réponse à Brian Warling

Re: Responses to a single question missing from quiz results for all students

par Emma Richardson,
Avatar Documentation writers Avatar Particularly helpful Moodlers Avatar Plugin developers

About has to be something with the question - perhaps it was set to allow multiple attempts and they did not complete them all or it could just be corrupted somehow...

Moyenne des évaluations  -
En réponse à Brian Warling

Re: Responses to a single question missing from quiz results for all students

par Jean-Michel Védrine,

Hello Brian,

I think something is broken in this question that prevent it from working.

Maybe it was broken during 1.9 -> 2.x -> 2.4 upgrade ?

To find what is broken you need some knowledge of how Moodle works and to look in the database, but you can make some tests:

- activate debugging so that all errors are displayed on screen (set it to DEVELOPER)

- What happen when you review some student's attempt for this question ?

- Going in the question bank, are you able to preview that question ? Are any error shown during preview if you submit a right or wrong answer ?

- Are you able to edit that question ? Using he HTML button of the editor is there some strange HTML for the question text or the answers ?

To solve your immediate problem, I suggest to change the grade for this question to 0 in the quiz and to regrade all attempts, then change the question's name so that it is no more used in future quizzes (something like DO NOT USE THIS QUESTION) and eventually recreate a new one with the same informations

Moyenne des évaluations  -
En réponse à Brian Warling

Re: Responses to a single question missing from quiz results for all students

par Tim Hunt,
Avatar Core developers Avatar Documentation writers Avatar Particularly helpful Moodlers Avatar Peer reviewers Avatar Plugin developers

That is certainly very weird.

In addition to what Emma and Jean-Michel suggest, here are some more things to try if you can:

  1. Go to the question bank, preview that question, and see if it works.
  2. Preview the whole quiz yourself, and try answering that question. Does your answer and mark show up on the review screen?
  3. Create a new quiz called something like Test quiz, in your course. Add this problem question. Log in with a test student account, and attempt it. Look in the quiz reports. Does the student's grade show up there?
  4. Poke around in the database. Find this quiz in the mdl_quiz and mdl_quiz_attempts tables, and see if it the data looks OK. Use this query: http://docs.moodle.org/dev/Overview_of_the_Moodle_question_engine#Detailed_data_about_an_attempt to look at the detailed attempts data.
  5. If you learn anything, let us know.

Sorry you hit this problem.

Moyenne des évaluations  -
En réponse à Tim Hunt

Re: Responses to a single question missing from quiz results for all students

par Brian Warling,

Thanks everyone for the quick and helpful replies. I can provide a bit more info (I should have provided this in my original post).

I did preview the question and it appears to work fine. Looking and the question itself, nothing appears out of the ordinary. It's built like all the other questions (multiple choice) in the quiz. I was able to preview the quiz with no problems. In the General Feedback, a simple table is used (2 columns, 1 row). Buy there are other questions that used tables in the feedback that didn't cause any problems.

We came across this exact same problem in a different quiz in August. I imported the quiz into a new course, logged into the course as a student, and took the quiz. The problem did not come up.... I was able to answer the question and the result was reported in the quiz results page. I did the same for the quiz I mentioned here -- I imported it into a new course, logged in as a student and took the quiz. The 58th question's answer was saved and was reported in the quiz results page.

My gut tells me that something happened in migrating this quiz from 1.9 to 2.x (and once it was migrated to 2.x, I think it was imported again into another 2.x course). Although it is interesting that when I imported this quiz into another course, the problem didn't appear again.

I think we'll poke around the database next. I'll report back what we learn.

Thanks again.... Brian

 

 

Moyenne des évaluations  -
En réponse à Brian Warling

Re: Responses to a single question missing from quiz results for all students

par Brian Warling,

It turns out that the questions and quiz were not migrated from a Moodle 1.9 course. They were built in Moodle 2.4. So I think we can remove the "migrated quiz" issue from the list of reasons why this problem happened.

We've been looking at the database. Below, I've included the two entries for a random attempt for the question we're having a problem with. This is the only question in the quiz where the final entry includes "gaveup" as the state. I'm not sure if what I've included below is relevant or helpful. Does this tell us anything? Is there other data that would be more helpful?

| 44430 | 1801 | 11178 | 1 | 54.00000 | deferredfeedback | 65 | deferredfeedback | 143908 | 1.0000000 | 0.0000000 | 0 | 0 | todo | NULL | 2013-09-20 09:17:41 | 44430 | _order | 550924,550922,550926,550925,550923 | 58. The MOST accurate description of the way in which pyridostigmine prevents severe effects of an organophosphate is by:: Inducing the P450 enzymes that metabolize organophosphates; Binding to the organophosphate and inactivating it; Serving as a competitive pharmacologic antagonist of muscarinic cholinergic receptors; Preventing the organophosphate from forming a long-lived covalent bond with cholinesterase enzyme; Increasing the amount of acetylcholine available to compete with organophosphate for binding to the nicotinic cholinergic receptors | Preventing the organophosphate from forming a long-lived covalent bond with cholinesterase enzyme | NULL |

| 44430 | 1801 | 11178 | 1 | 54.00000 | deferredfeedback | 65 | deferredfeedback | 143908 | 1.0000000 | 0.0000000 | 0 | 1 | gaveup | NULL | 2013-09-20 11:34:18 | 44430 | -finish | 1 | 58. The MOST accurate description of the way in which pyridostigmine prevents severe effects of an organophosphate is by:: Inducing the P450 enzymes that metabolize organophosphates; Binding to the organophosphate and inactivating it; Serving as a competitive pharmacologic antagonist of muscarinic cholinergic receptors; Preventing the organophosphate from forming a long-lived covalent bond with cholinesterase enzyme; Increasing the amount of acetylcholine available to compete with organophosphate for binding to the nicotinic cholinergic receptors | Preventing the organophosphate from forming a long-lived covalent bond with cholinesterase enzyme | NULL |

*******

As I mentioned earlier, when I imported the quiz into a new course and then attempted the quiz (as a student), everything went fine. The answer to the problematic question was recorded. Also, when I previewed the quiz in the original course, the answer to this question was reported back.

And in case if it's any help, I've attached the XML of the problematic question. It looks OK to me.

Thanks again for any help.  -- Brian

Moyenne des évaluations  -
En réponse à Brian Warling

Re: Responses to a single question missing from quiz results for all students

par Tim Hunt,
Avatar Core developers Avatar Documentation writers Avatar Particularly helpful Moodlers Avatar Peer reviewers Avatar Plugin developers

That data is entirely consisten with a quiz where

  1. The student clicked start attempt.
  2. They did not input a response for this quesion.
  3. They clicked submit all and finish.

That is what 'gaveup' means. I means no (gradable) response given, as opposed to incorrect response given.

Moyenne des évaluations  -
En réponse à Tim Hunt

Re: Responses to a single question missing from quiz results for all students

par Brian Warling,

Tim..... Yes, I see what you're saying. But we don't think that 150 medical students would have all followed that same path for only that question in a 60 question exam. Something else must be going on. We will keep looking for that (I hope to have an epiphany soon sourire.

Moyenne des évaluations  -
En réponse à Brian Warling

Re: Responses to a single question missing from quiz results for all students

par Tim Hunt,
Avatar Core developers Avatar Documentation writers Avatar Particularly helpful Moodlers Avatar Peer reviewers Avatar Plugin developers

Look at quiz_attempts.layout column, and compare it to quiz.questions and quiz_question_instances. Wild guess, but it might show something.

Moyenne des évaluations  -
En réponse à Tim Hunt

Re: Responses to a single question missing from quiz results for all students

par Brian Warling,

We actually received more information from the faculty about this exam. It turns out that the students were able to see this question, and they answered it. Moodle dutifully recorded the responses. The faculty downloaded the exam responses to Excel.

But they wanted to remove the question after the exam since they didn't cover this topic in class. They first edited the question, marking all answers as correct (worth 100%), then regraded the exam. The responses were still there. They then decided to remove the question from the quiz score entirely -- they changed the question back so that only one answer was correct, and changed the question mark to 0, then regraded. After doing this (according to them), all the responses are gone for that question.

I tried to replicate this on my test version of this quiz. When finished, there was no grade for this question, but there is a full response history (see image below). In the actual quiz, the response history only has "Started" and "Attempt finished" for that question (see the image in my initial post).

Now that we know that Moodle did record the response to this question, and the faculty did mess with this question and regrade (more than once) -- and yes, I agree that it would have been much better if they had been upfront about this at the start -- but the issue is now different.  Is there a workflow that could be followed after a quiz that would cause a question's response to disappear?

Many thanks again for all the help.... Brian

Moyenne des évaluations  -
En réponse à Brian Warling

Re: Responses to a single question missing from quiz results for all students

par Tim Hunt,
Avatar Core developers Avatar Documentation writers Avatar Particularly helpful Moodlers Avatar Peer reviewers Avatar Plugin developers

I can't see how anything the faculty did could have caused Moodle to lose the responses.

If, after a quiz, you don't want one question to count, the best way to do that is to change the Maximum mark for that question to 0 on the edit quiz page. You don't even have to regrade. The total scores are immediately re-computed when you make that change. Also, the quiz review will show 'Not graded' (or something similar) for that question in place of the grade. You will still be able to see the student's response.

But changing the grades for multiple choice choices and regrading is also safe.

Unsafe would be:

  • Editing a multiple choice question to add or remove choices.
  • If you use Interactive with multiple tries behaviour, which stops after the first right answer, then editing a multiple choice question to change the grade for each choice, and regrading, can lose the second and subsequent responses. (One day I intend to fix that.)
Moyenne des évaluations  -
En réponse à Tim Hunt

Re: Responses to a single question missing from quiz results for all students

par Mary Beth McKee,

We have used Moodle for several years now, but yesterday for the first time, I saw where a teacher edited two multiple choice questions after students had taken the quiz, done a regrade, and as Brian described, all the student responses to those two edited questions were deleted.

We are using Moodle 2.5.3+ (Build: 20131129)

I had never seen seen student responses removed for a single question like that before, so I went to our automated backup system and restored an earlier copy of the quiz (before the questions were edited) to a test class, and the student responses did exist in the backup file.

Description of Events

  1. Teacher created a timed quiz allowing a single attempt.
  2. Teacher discovered two bad questions in the quiz.
  3. She edited the questions twice and saved.
    One edit involved changing the question value from one to zero.
    The second edit involved changing the multiple choice from single answer to multiple answer, changing the various percentage weights assigned to the answers. Returned the question weight to one.
  4. At this point the teacher called me, and I had her change the multiple choice question back to single answer, but gave two of the answers a weight of 100%
  5. Then the teacher did a regrade.
  6. All of the students responses to the two edited questions were deleted. The other quiz questions were intact.
  7. When I restored a backup of her course to a test course, I could see where the student responses to those questions did exist.

Mary Beth

 

Moyenne des évaluations  -
En réponse à Mary Beth McKee

Re: Responses to a single question missing from quiz results for all students

par Tim Hunt,
Avatar Core developers Avatar Documentation writers Avatar Particularly helpful Moodlers Avatar Peer reviewers Avatar Plugin developers

The only way I can see that would cause what you describe is if the teacher did a regrade after Step 3. That would delete the responses to the multiple choice question.

Changing the question value (maximum mark / marked out of) is completely safe. It does not even require a regrade. The scores are re-calculated immediately.

Changing some elements of a question, e.g. the grade of each choice in a multiple choice test, or correcting a typo in the question text, is safe, but does require a regrade.

Changing the type of a question: don't do that.

Moyenne des évaluations  -
En réponse à Tim Hunt

Re: Responses to a single question missing from quiz results for all students

par Mary Beth McKee,

Hi Tim,

Thanks for your reply.  Sometimes our faculty try to fix things before calling the help line.  The faculty member knew she wanted to make two multiple choices correct, so she switched from single answer to multiple answer, adjusted percentages, and did a regrade. In her head, she wanted multiple answers to be correct, so she figured toggling that setting would make sense.

In testing, I found that changing the question type in this way does delete the student answers.  I appreciate the help.

Mary Beth

Moyenne des évaluations  -
En réponse à Mary Beth McKee

Re: Responses to a single question missing from quiz results for all students

par Karla Bright,

Hi Mary Beth,

We have had this same problem just this past week.  Thanks so much for posting. 

Our professor did just what you described and now the student's answers to that question have been deleted.  I don't know how to fix it, but it helps to know that it is a "bug" in the system.

Karla Bright



Moyenne des évaluations  -
En réponse à Karla Bright

Re: Responses to a single question missing from quiz results for all students

par Susana L.,

Hi,

Just to say that we had a similar problem. A teacher changed from multiple answer to single answer and regraded the quiz...
susana 

Moyenne des évaluations  -