Calculated questions not marking correctly.

Calculated questions not marking correctly.

by Mark Zeits -
Number of replies: 24
Hello,
I have a Moodle set up (version 1.5.2-latest) I believe the build was a couple days ago. I have been having intermittent problems with calculated quesitons and students submitting what seems to be correct answers and coming back incorrect. Here is an example.  When editing the quiz and submitting while previewing it, the quiz seems to work correctly.
Sample submission.
and here is the actual quiz settings...
Quiz settings...

one more example...
Second Submission Example
and the quiz settings...
Second Quiz Settings

am I just missing something simple??? Is this a known bug? Thanks for your help!!

-mark
Average of ratings: -
In reply to Mark Zeits

Re: Calculated questions not marking correctly.

by Orestes Mas -
Picture of Translators
Astounding! I'm having similar problems and I started to believe I'm turning crazy... In a recent quiz, all students got a grade of 0 to the only one calculated question in the quiz (but some responses were correct).

Here's my question (in Catalan language, but the contents aren't important in this case):

Question

As you can see, the student's answer is 2,3V, which is correct (the Catalan locale uses the comma as decimal separator, but Moodle accepts either the period and the comma, I'v tested it)

The question settings are:

settings

Regrading has no effect on this question (the last one shows no hyperlinked text). Curiously, there are other questions showing hyperlinked attempts, indicating attempts that change during regrade, but there aren't changed attempts at all. These occurs every time I regrade.

Regrading

Look for example to the attempt number 265 in the last question (the one with a red circle around). I've searched for it into the database (using the phpMyAdmin module for Moodle) and the results are:

database

i.e.: The student answer is correctly stored.


In short: I think this is a serious bug affecting the student's grading, and I would be very pleased if someone points me in the good direction to solve it.

Ops! I forgot to say that I'm using Moodle 1.5.2 + (2005060223)

Thanks in advance.

Average of ratings: -
In reply to Mark Zeits

Re: Calculated questions not marking correctly.

by Orestes Mas -
Picture of Translators
I think I found it:

The problem seems related to the "tolerance" factor. Although I think my tolerance is correct, because with a tolerance of 0.02 (relative), Moodle shows a range of valid responses between 2.254 and 2.346. The answer of the student was 2.3, so it was ok, but graded erroneously.

Then I increased the tolerance to 0.2 (ten times), and now Moodle grades this answer correctly. The range of possible answers now goes from 1.84 to 2.76, which also includes 2.3, obviously.

Anyone with deep quiz code knowledge can help?
Average of ratings: -
In reply to Orestes Mas

Re: Calculated questions not marking correctly.

by Jean-Michel Védrine -
In all problems with calculated questions it is interesting to look at the item analysis report for this question because for each student response, the tolerance interval is displayed.
I must admit I had a lot of problems with relative tolerance and now I always choose an absolute one.
Average of ratings: -
In reply to Jean-Michel Védrine

Re: Calculated questions not marking correctly.

by Orestes Mas -
Picture of Translators
For me, Moodle only displays the tolerance for the wrong responses. For the right ones, the formula to calculate them is displayed instead.

answers

The same to you?

Anyway, I'll try the absolute tolerance in the future, although in this case (and many others) a relative one is more adequate.
Average of ratings: -
In reply to Orestes Mas

Re: Calculated questions not marking correctly.

by Jean-Michel Védrine -
Yes you are right, and the funny thing is that it was in fact one of my own requests to Enrique : to group all the right answers in one line because when you have a lot of students it was very uneasy to have a line for each correct result if it was rounded differently.
As consequence we can't display the tolerance intervall because it is different for each member of the dataset.
I must add that if you install my detailed responses report plugin (available in that thread) right answer and tolerance are displayed for each student response grand sourire
As a math teacher absolute tolerance is working for me in nearly all situations but I will look in the relative tolerance code to see if I find any bug.
Average of ratings: -
In reply to Jean-Michel Védrine

Re: Calculated questions not marking correctly.

by Jean-Michel Védrine -
After some tests, I can say the problem is indeed more serious because all tolerance types are affected, not only the relative one. The problem seems to happend after using the "Regrade attempts" tab : all attempts seems to be regraded with the answer for the first attempt, not the right one for the actual attempt. All these attempts were graded correctly before using "Regrade attempts" tab.
My "detailed responses" report plugin is also affected by this bug because all attempts show the right answer and tolerance for the first attempt, but for the moment I am thinking this is only because of the bug in the calculated questiontype code.
I will work on the problem ...

Average of ratings: -
In reply to Jean-Michel Védrine

Re: Calculated questions not marking correctly.

by Jean-Michel Védrine -
Good news : data in the database seems corrects so once we find the bug, regrading all calculated questions should correct all wrong grades !
Average of ratings: -
In reply to Jean-Michel Védrine

Re: Calculated questions not marking correctly.

by Julian Sedding -
Jean-Michel,

if you need any support with this, feel free to contact me. I have tried to replicate this problem myself, but it works fine for me. If you can give me a clue on how to replicate the bug, I'll be happy to investigate again.

Julian
Average of ratings: -
In reply to Julian Sedding

Re: Calculated questions not marking correctly.

by Julian Sedding -
After an enlightening email from Jean-Michel containing his findings and a guide to replicate the bug, I think I could solve the problem. I changed the file /mod/quiz/reports/regrade/report.php and committed it to CVS 1.5 stable.

Mark and Orestes, could you please upgrade this file, regrade your affected quizzes and check whether this fix does indeed solve your problem? If you could just quickly confirm here, so I can close bug #4110, please!

@Orestes: The logic that determined whether the grading of a question or not failed, if the grade was a fraction (e.g. 0.33333). The value retrieved from the database was less precise than the calculated one and therefore the comparison indicated a change. Rounding both of these values does the trick and doesn't wrongly indicate that questions were regraded (that's also in CVS now).

Julian
Average of ratings: Useful (1)
In reply to Julian Sedding

Re: Calculated questions not marking correctly.

by Julian Sedding -
And another update: The change from the previous post is undone. Instead the calculated questiontype (and datasetdependent) have been modified to prevent this problem from happening in the first place.

The advantage: now the detailed responses report also works correctly.

Julian
Average of ratings: Useful (1)
In reply to Julian Sedding

Re: Calculated questions not marking correctly.

by Orestes Mas -
Picture of Translators
Many, many thanks!

But I've seen there's only minor changes in /mod/quiz/report/regrade/report.php (only one line and the date). I also see that you changed another file in CVS (/mod/quiz/locallib.php). I presume this last one is also needed.

I'll try them in a while.

Average of ratings: -
In reply to Orestes Mas

Re: Calculated questions not marking correctly.

by Julian Sedding -
The relevant changes for regrading are in /mod/quiz/questiontypes/calculated/questiontype.php and /mod/quiz/questiontypes/datasetdependent/abstracttype.php. The change in /mod/quiz/report/regrade/report.php was undone, because it was more of a workaround than a solution.

Additionally there is a change in locallib.php, but that is only to fix the problem where questions were shown as regraded that actually stayed unmodified.

Julian
Average of ratings: -
In reply to Julian Sedding

Re: Calculated questions not marking correctly.

by Orestes Mas -
Picture of Translators
Julian,

Partial success: As you point out, regrading no longer indicates not-regraded questions as being regraded. This is fixed.

Unfortunately, the relative tolerance still doesn't work for me. Same behaviour. And I've also noticed that In the "Overview" tab, when I check the "show mark details" checkbox, I can see there were 2 students (2 attempts total) that answered this question correctly. But in the "item analysis" tab the number of correct answers for this question is set to 4!!

If you point me to the lines of the source code where these calculations are performed, I can try to give help finding this bug...


Average of ratings: -
In reply to Orestes Mas

Re: Calculated questions not marking correctly.

by Jean-Michel Védrine -
The most important changes to solve this problem are in the mod/quiz/questiontypes/calculated/questiontype.php file. Did you upgrade to the last 15_STABLE version of this file ? On my 2 websites it solve the problem for all calculated questions I made to test the problem.
Average of ratings: -
In reply to Jean-Michel Védrine

Re: Calculated questions not marking correctly.

by Orestes Mas -
Picture of Translators
I'm sorry I didn't. The day I tried it I thought the changes were too recent to be already in the 15_STABLE build. I'll try it in a while. Thanks.

Average of ratings: -
In reply to Julian Sedding

Re: Calculated questions not marking correctly.

by Mark Zeits -

Julian,

  Thanks for the quick response!  I have been away from my email for a while, I will update my site tonight and let you know as soon as possible.

-mark

Average of ratings: -
In reply to Mark Zeits

Re: Calculated questions not marking correctly.

by Mark Zeits -
Julian,
  I upgraded all the files mentioned and went back to the offending quiz and tried "Regrade Attempts" and it is still marking right answers as incorrect.  I will let you know about any new quiz's that I am having the kids do to see if the tolerance problem is fixed.

-mark
Average of ratings: -
In reply to Mark Zeits

Re: Calculated questions not marking correctly.

by Julian Sedding -
Hi Mark,

if you're using PHP 4 I could replicate your problem. I asked Eloy to include an improvement to lib/moodlelib.php into the distribution that fixed the problem for me (I only have write access to the quiz module, so I can't commit this to CVS).

Please try replacing your lib/moodlelib.php with the attached version of the file and see if regrading works.

By the way, there was nothing wrong with the tolerance. You may have noticed that the first attempt in the list was always correctly regraded. Any subsequently regraded attempts then used the same dataset that was used for the first attempt, which obviously would result in a lot of "incorrect" responses.

Julian
Average of ratings: -
In reply to Julian Sedding

Re: Calculated questions not marking correctly.

by Mark Zeits -

Julian,

  After updating the moodlelib.php file, everything seems to be working correctly.  The calculated questions, as far as I can tell are working, I need more students to start the second quiz to find out.  The "Regrade Attempts" is also working too.  I will be watching the grades closely to see if anything else comes up...  Thanks much for helping!

-mark

Average of ratings: -
In reply to Julian Sedding

Re: Calculated questions not marking correctly.

by Eloy Lafuente (stronk7) -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Peer reviewers Picture of Plugin developers Picture of Testers
Hi,

Julian changes have been applied to Moodle 1.5 CVS (and HEAD). Hope it helps to solve this issue (without creating anything new wink).

Please test them in your sites...thanks!

Ciao smile
Average of ratings: -
In reply to Eloy Lafuente (stronk7)

Re: Calculated questions not marking correctly.

by Orestes Mas -
Picture of Translators
With the updated moodlelib.php now regrading is working fine. Thank you bug-busters smile
Average of ratings: -
In reply to Orestes Mas

Re: Calculated questions not marking correctly.

by Fig Newton -
I have run into the same problem regrading calculated questions. I have installed the lastest stable build for quiz/questiontypes/*.php and the moodlelib.php file posted above. My regrades attempts are *still* not being marked correctly.

-Fig
Average of ratings: -