Any help on this would be appreciated.
~Cynthia
Quiz Total Doesn't Match Earned Points
Number of replies: 14
I have an instructor who just gave a quiz. Several students were told they had one score (i.e., 14 correct out of 15) when they finished the quiz and when looking at the quiz results, those scores are displayed. However, when looking at the questions missed and what is reported to the grade book, it indicates that the students got 15 correct out of 15. I checked the quiz set up and only one point is given to each question. I also checked each question to make sure that the answer is given only one correct answer, that it is worth one point, and that there is only one answer for each question that indicates 100% and the rest of the options indicate None. This is a multiple choice test and we are using 1.9.2.
Any help on this would be appreciated.
~Cynthia
Any help on this would be appreciated.
~Cynthia
Re: Quiz Total Doesn't Match Earned Points
You are not the first person to see this sort of thing. It is very strange though because it only happens occasionally, and I have never observed it happening in my development set-up, which makes it almost impossible for me to diagnose and fix. So while this worries and puzzles me, there is very little I can do.
Re: Quiz Total Doesn't Match Earned Points
Tim -
Unfortunately, this will mean that the teacher will have to explain this to the students. I understand your predicament, but we are a very large community college (student population of over 25,000) that has a total of 11 online courses. My job here is to encourage faculty to 1) use Moodle and then 2) to move to at least some online courses. The teacher will, unfortunately, be telling other faculty that the system isn't "stable".
~Cynthia
Unfortunately, this will mean that the teacher will have to explain this to the students. I understand your predicament, but we are a very large community college (student population of over 25,000) that has a total of 11 online courses. My job here is to encourage faculty to 1) use Moodle and then 2) to move to at least some online courses. The teacher will, unfortunately, be telling other faculty that the system isn't "stable".
~Cynthia
Re: Quiz Total Doesn't Match Earned Points
Tim -
Sorry. But one more question on this. What IS the correct grade??? Is it the amount reported (i.e., 14) or the missed showing in the list of questions (i.e., 1/1, 1/1, 1/0, etc.)?
~Cynthia
Sorry. But one more question on this. What IS the correct grade??? Is it the amount reported (i.e., 14) or the missed showing in the list of questions (i.e., 1/1, 1/1, 1/0, etc.)?
~Cynthia
Re: Quiz Total Doesn't Match Earned Points
Tim -
I've talked this over with my colleagues here at the college. I have two questions. 1) Was this a problem with Moodle prior to 1.9? 2) If I backed up the quiz in which this problem occurs without student data, would it be something that you could work with to possibly remedy?
~ Cynthia
I've talked this over with my colleagues here at the college. I have two questions. 1) Was this a problem with Moodle prior to 1.9? 2) If I backed up the quiz in which this problem occurs without student data, would it be something that you could work with to possibly remedy?
~ Cynthia
Re: Quiz Total Doesn't Match Earned Points
My impression is that problems like this did not occur (or at least not so frequently) before Moodle 1.9. However, this bit of the quiz did not change very much between 1.8 and 1.9, so I don't understand what caused things to change. (The impression just comes from my memory of what people post in this forum and in the tracker.)
A backup without user data probably won't contain the necessary information, unfortunately. (We need to create the facility to do a Moodle backup while randomising the sensitive data like student names - that is one of those things that developers discuss from time to time, but never get around to.)
My understanding from people who have reported this problem before is that the individual question scores are right - the error is in adding up the individual scores to get the overall total. Anyway, the error will be systematic (the same for all attempts) so if you review a few attempts, it should be clear what the answer is.
Thinking about how this bit of the code works. It is one of those things in the quiz that I have never been entirely happy with, but have not yet had sufficient cause to change. The quiz attempt score starts at zero, and as the student answers questions and gets scores, the attempt score in adjusted up and down - and I think it does that even if you rescore. So once the attempt score gets messed up, it stays messed up.
I fear the time may have come where I have to address this. It would not be hard (I think) to change regrading so that at the end of the process, it re-computes the attempt scores from scratch. However, rather than change regrading so the problem can be fixed after it has happened, it would be better to find out what goes wrong to cause the inconsistency in the first place.
And I do understand that for the few people that this affects, it id disastrous for Moodle's reputation and uptake.
A backup without user data probably won't contain the necessary information, unfortunately. (We need to create the facility to do a Moodle backup while randomising the sensitive data like student names - that is one of those things that developers discuss from time to time, but never get around to.)
My understanding from people who have reported this problem before is that the individual question scores are right - the error is in adding up the individual scores to get the overall total. Anyway, the error will be systematic (the same for all attempts) so if you review a few attempts, it should be clear what the answer is.
Thinking about how this bit of the code works. It is one of those things in the quiz that I have never been entirely happy with, but have not yet had sufficient cause to change. The quiz attempt score starts at zero, and as the student answers questions and gets scores, the attempt score in adjusted up and down - and I think it does that even if you rescore. So once the attempt score gets messed up, it stays messed up.
I fear the time may have come where I have to address this. It would not be hard (I think) to change regrading so that at the end of the process, it re-computes the attempt scores from scratch. However, rather than change regrading so the problem can be fixed after it has happened, it would be better to find out what goes wrong to cause the inconsistency in the first place.
And I do understand that for the few people that this affects, it id disastrous for Moodle's reputation and uptake.
Re: Quiz Total Doesn't Match Earned Points
Just another data point.
I am using moodle 1.9.1+ and I have quizzes set to adaptive mode, no penalty. Because of this, students can submit each individual question. (These are webwork questions, not m/c, so I allow them to resubmit as often as need be until it closes). Unless the student clicks on "submit all and finish", the grades are not calculated. Thus, I've had students take the quiz more than once. The final time, they had everything correct, but they forgot to click the submit all button. Thus, the grades will show that they have every question correct, but the overall grade will be too low. I also see no grade for students who submitted problems individually, but never officially finished the quiz.
I don't know if this is related to your problem (you didn't mention the parameters of the quiz) - it's probably not, but you never know.
It would be nice if regrade would be able to recognize if a quiz has closed and then post grades for these open, but stranded, quizzes. I've looked at the code and I think it should be able to do it, but I'm not a programmer by profession.
It would be even nicer if at closing time, the stranded quizzes were graded automatically, but I imagine that would be a harder programming problem, so I'm not holding my breath
I am using moodle 1.9.1+ and I have quizzes set to adaptive mode, no penalty. Because of this, students can submit each individual question. (These are webwork questions, not m/c, so I allow them to resubmit as often as need be until it closes). Unless the student clicks on "submit all and finish", the grades are not calculated. Thus, I've had students take the quiz more than once. The final time, they had everything correct, but they forgot to click the submit all button. Thus, the grades will show that they have every question correct, but the overall grade will be too low. I also see no grade for students who submitted problems individually, but never officially finished the quiz.
I don't know if this is related to your problem (you didn't mention the parameters of the quiz) - it's probably not, but you never know.
It would be nice if regrade would be able to recognize if a quiz has closed and then post grades for these open, but stranded, quizzes. I've looked at the code and I think it should be able to do it, but I'm not a programmer by profession.
It would be even nicer if at closing time, the stranded quizzes were graded automatically, but I imagine that would be a harder programming problem, so I'm not holding my breath
Re: Quiz Total Doesn't Match Earned Points
The problem you are seeing is a different issue. One that has been discussed before, but never adequately resolved.
Re: Quiz Total Doesn't Match Earned Points
OK, I think I have worked out what we would need to do to compute the score afresh each time, instead of adjusting it incrementally. However, I would, of course, like to be really sure I have got the right formula! Therefore it would be helpful if people with direct access to their database could run the following query:
SELECT qa.id AS quizattempt, sess.attemptid, SUM(state.grade) AS computedgrade, qa.sumgrades AS storedgrade
FROM mdl_question_sessions sess
JOIN mdl_question_states state ON state.id = sess.newgraded
JOIN mdl_quiz_attempts qa ON qa.uniqueid = sess.attemptid
GROUP BY attemptid, qa.sumgrades
HAVING SUM(state.grade) <> qa.sumgrades
What this does is to compare the stored grade for each quiz with the result of applying what I think is the correct formula. So:
1. If you have never had problems with Moodle computing the score for attempts, no records should be returned. (Or only records where the two scores differ because of rounding errors.)
2. If you have a site where problems like the one discussed in this thread have occurred, this query should list just the problem attempts. You can double check that by going to .../mod/quiz/review.php?attempt=[quizattempt]
The more people who can confirm that the formula is right, the better. Thanks.
SELECT qa.id AS quizattempt, sess.attemptid, SUM(state.grade) AS computedgrade, qa.sumgrades AS storedgrade
FROM mdl_question_sessions sess
JOIN mdl_question_states state ON state.id = sess.newgraded
JOIN mdl_quiz_attempts qa ON qa.uniqueid = sess.attemptid
GROUP BY attemptid, qa.sumgrades
HAVING SUM(state.grade) <> qa.sumgrades
What this does is to compare the stored grade for each quiz with the result of applying what I think is the correct formula. So:
1. If you have never had problems with Moodle computing the score for attempts, no records should be returned. (Or only records where the two scores differ because of rounding errors.)
2. If you have a site where problems like the one discussed in this thread have occurred, this query should list just the problem attempts. You can double check that by going to .../mod/quiz/review.php?attempt=[quizattempt]
The more people who can confirm that the formula is right, the better. Thanks.
Re: Quiz Total Doesn't Match Earned Points
Tim -
Unfortunately, our server is hosted offsite by Remote Learner so it is impossible for me to access the database and difficult to have Remote Learner do this for us. I hope others will be able to "pitch in" and help out.
~Cynthia
Unfortunately, our server is hosted offsite by Remote Learner so it is impossible for me to access the database and difficult to have Remote Learner do this for us. I hope others will be able to "pitch in" and help out.
~Cynthia
Re: Quiz Total Doesn't Match Earned Points
Tim, et al -
We finally figured it out (just about the same time Remote Learner also figured it out). The teacher had the quiz set up in adaptive mode with penalties. I am sorry for all of the headache.
~Cynthia
We finally figured it out (just about the same time Remote Learner also figured it out). The teacher had the quiz set up in adaptive mode with penalties. I am sorry for all of the headache.
~Cynthia
Re: Quiz Total Doesn't Match Earned Points
Brilliant. It reminds me of all those times when a problem is reported and I go into to code and actually find an embarrassing mistake in the code and fix it to my satisfaction only to find out that the reported "problem" is in fact a normal behavior upon certain settings and I'm left wondering what exactly I've just fixed. 
Re: Quiz Total Doesn't Match Earned Points
No worries. Thank you, and Remote Learner for taking the time to investigate.
Re: Quiz Total Doesn't Match Earned Points
Hi Cynthia,
I'm using 1.9.2+ and quizzes quite intensively and as far as I can tell I haven't encountered this problem. One difference between my quizzes and the one in the image above is that my quizzes' grade is not a simple sum of the questions' marks but rather normalized to 100. So perhaps there is a bug which occur in simple sums and perhaps only in low numbers. I don't know. It may be worthwhile to try changing the quiz's grade to 100 or some other number and see how the total is calculated.
Itamar
I'm using 1.9.2+ and quizzes quite intensively and as far as I can tell I haven't encountered this problem. One difference between my quizzes and the one in the image above is that my quizzes' grade is not a simple sum of the questions' marks but rather normalized to 100. So perhaps there is a bug which occur in simple sums and perhaps only in low numbers. I don't know. It may be worthwhile to try changing the quiz's grade to 100 or some other number and see how the total is calculated.
Itamar
Re: Quiz Total Doesn't Match Earned Points
Itamar -
Thanks for the suggestion, but it is not my test. We are a fairly large community college with only 21 online sections and 19 hybrid sections. My job is to help encourage faculty to move to online and give support with Moodle. The college just switched to Moodle recently (from Etudes) and not all faculty who had been using the old system have been receptive to the change. The suggestion to change the quiz grading will not be met with enthusiasm.
~Cynthia
Thanks for the suggestion, but it is not my test. We are a fairly large community college with only 21 online sections and 19 hybrid sections. My job is to help encourage faculty to move to online and give support with Moodle. The college just switched to Moodle recently (from Etudes) and not all faculty who had been using the old system have been receptive to the change. The suggestion to change the quiz grading will not be met with enthusiasm.
~Cynthia