In a calculated question (Moodle 1.9), it seems to me that an answer is marked as "Correct" if it falls within the tolerance band of the exact answer. However, it is marked with a green tick if it falls within the tolerance band of the rounded answer (i.e. the answer rounded to the required number of, for example, significant figures).
On occasion, this leads to an answer marked as "Correct" but given a red cross and, at other times, to an answer marked as "Incorrect" but given a green tick. The frequency of occurrence can be minimised by increasing the number of sig figs but I wonder why this is how things are set up. What advantage does this offer that outweighs the confusion engendered when it's first encountered?
I'm new to this, by the way, and find Moodle impressive but I couldn't find the answer to my query after digging round for a bit.
Graham
"Correct" and a red cross can appear simultaneously in a calculated question. Why have it so?
by graham crowder -
Number of replies: 9
In reply to graham crowder
Re: "Correct" and a red cross can appear simultaneously in a calculated question. Why have it so?
by Tim Hunt -
Can you give us a specific example of a question (Export it as Moodle XML perhaps, or just tell us the exact settings for creating it) and the answer you need to type to see this problem. Even better, put that information into the Moodle bug tracker: http://tracker.moodle.org/
In reply to Tim Hunt
Re: "Correct" and a red cross can appear simultaneously in a calculated question. Why have it so?
by graham crowder -
Thanks, Tim.
Here's how to generate the situation. I'll look into how to produce XML and I'm happy to submit the bug report but thought I'd check first that it wasn't my lack of experience shining through.
1. Set up a calculated question such as "What is 2 times {A}?" with a formula of 2*{A}
2. Use defaults of 0.01 relative tolerance and set correct answer format of 3 sig fig. (I also set grade to 100%)
3. Set up a single answer item with wild card value for A of 1.234
The correct answer is then shown as 2.47 with min 2.4453 and max 2.4947.
Note that the exact answer is 2.468 with min 2.44332 and max 2.49268.
The following submitted values in the preview produce the responses shown for Correct/Incorrect and Tick/Cross:
2.49 Correct + Tick
2.493 Incorrect + Tick
2.495 Incorrect + Cross
2.446 Correct + Tick
2.444 Correct + Cross
2.440 Incorrect + Cross
Looking at this set of responses, it seems to me that "Correct" is applied when the submitted answer is in the tolerance range of the exact answer and a "Tick" is applied when the submitted answer is in the tolerance range of the rounded answer, i.e. the answer Moodle views as correct having rounded to the specified 3 sig fig.
In my view, the Tick/Cross should be applied exactly as for the Correct/Incorrect response, with the rounded "correct" answer simply being shown as the correct answer on demand. If there's a good reason for doing it as is then it has to outweigh the confusion that students, or novice Moodle instructors like me, will feel when an answer marked as Correct is awarded a red Cross, and vice versa.
Hope that all makes sense.
Graham
Here's how to generate the situation. I'll look into how to produce XML and I'm happy to submit the bug report but thought I'd check first that it wasn't my lack of experience shining through.
1. Set up a calculated question such as "What is 2 times {A}?" with a formula of 2*{A}
2. Use defaults of 0.01 relative tolerance and set correct answer format of 3 sig fig. (I also set grade to 100%)
3. Set up a single answer item with wild card value for A of 1.234
The correct answer is then shown as 2.47 with min 2.4453 and max 2.4947.
Note that the exact answer is 2.468 with min 2.44332 and max 2.49268.
The following submitted values in the preview produce the responses shown for Correct/Incorrect and Tick/Cross:
2.49 Correct + Tick
2.493 Incorrect + Tick
2.495 Incorrect + Cross
2.446 Correct + Tick
2.444 Correct + Cross
2.440 Incorrect + Cross
Looking at this set of responses, it seems to me that "Correct" is applied when the submitted answer is in the tolerance range of the exact answer and a "Tick" is applied when the submitted answer is in the tolerance range of the rounded answer, i.e. the answer Moodle views as correct having rounded to the specified 3 sig fig.
In my view, the Tick/Cross should be applied exactly as for the Correct/Incorrect response, with the rounded "correct" answer simply being shown as the correct answer on demand. If there's a good reason for doing it as is then it has to outweigh the confusion that students, or novice Moodle instructors like me, will feel when an answer marked as Correct is awarded a red Cross, and vice versa.
Hope that all makes sense.
Graham
In reply to graham crowder
Re: "Correct" and a red cross can appear simultaneously in a calculated question. Why have it so?
by Pierre Pichet -
The problem is related to a possible conflict between the significant figures allowed and tolerance range which is 0.01 i.e 1%.
I am working on an message telling the user that there is a possible conflict between the two.
See MDL-14433 and the related discussion http://moodle.org/mod/forum/discuss.php?d=87939
However I did not see those variations between error display and Correct answer that you pinpoint.
The Correct response is not displayed in the same code that displays the colors. This remember me something. I check and come back .
Pierre
P.S. "soon" (see last comment on MDL-14433 20 april 2008) remains you that time goes by rapidly
.
I am working on an message telling the user that there is a possible conflict between the two.
See MDL-14433 and the related discussion http://moodle.org/mod/forum/discuss.php?d=87939
However I did not see those variations between error display and Correct answer that you pinpoint.
The Correct response is not displayed in the same code that displays the colors. This remember me something. I check and come back .
Pierre
P.S. "soon" (see last comment on MDL-14433 20 april 2008) remains you that time goes by rapidly
.
In reply to graham crowder
Re: "Correct" and a red cross can appear simultaneously in a calculated question. Why have it so?
by Pierre Pichet -
The feedback image and color and cross or check are setted by the numerical question print_question_formulation_and_controls using the 3 significant figure and the tolerance settings.
However the Correct and correct response are setted by the numerical grade_responses using the exact mathematical solution without the significant figures so that the correct answer is the correct mathematical value.
We can introduce a message for the student that explain the difference between the grading and the correct answer and synchronize the correct-incorrect message about his response.
This can be worked in MDL-14433 although this is not simple as actually the numerical question grade_responses() is used i.e. we need to create a specific calculated grade_responses() function.
Actually the calculated has numerical question as a virtual->qtype.
I will look and comment the technical details on the tracker
Pierre
However the Correct and correct response are setted by the numerical grade_responses using the exact mathematical solution without the significant figures so that the correct answer is the correct mathematical value.
We can introduce a message for the student that explain the difference between the grading and the correct answer and synchronize the correct-incorrect message about his response.
This can be worked in MDL-14433 although this is not simple as actually the numerical question grade_responses() is used i.e. we need to create a specific calculated grade_responses() function.
Actually the calculated has numerical question as a virtual->qtype.
I will look and comment the technical details on the tracker
Pierre
In reply to Pierre Pichet
Re: "Correct" and a red cross can appear simultaneously in a calculated question. Why have it so?
by Pierre Pichet -
"the numerical question grade_responses() is used"
On a closer look there is no numerical grade_responses(), the default question one is used.
So the difference remain because in printing the answer feedback the values use the significant figures or decimal points and on grading only the true mathematical value is used.
We need to use the true value on grading with the tolerance limits set by the teacher.
The significant figures or decimals should be set as large as necessary to express all values within the tolerance limit. Doing this the tolerance limit will be the control factor and the correct-incorrect will synchronize.
Pierre
On a closer look there is no numerical grade_responses(), the default question one is used.
So the difference remain because in printing the answer feedback the values use the significant figures or decimal points and on grading only the true mathematical value is used.
We need to use the true value on grading with the tolerance limits set by the teacher.
The significant figures or decimals should be set as large as necessary to express all values within the tolerance limit. Doing this the tolerance limit will be the control factor and the correct-incorrect will synchronize.
Pierre
In reply to graham crowder
Re: "Correct" and a red cross can appear simultaneously in a calculated question. Why have it so?
by Pierre Pichet -
The problem you pinpoint comes to student notice because you did not allow the number of significant figure to express correctly the exact mathematical response.
i.e. 2.468 needs 4 significant figures.
This is a quite an unusual condition (people set number of decimals more often) so it has not be reported before as far as I remember
However your observations are correct and the code will be worked to correct it
.
Pierre
i.e. 2.468 needs 4 significant figures.
This is a quite an unusual condition (people set number of decimals more often) so it has not be reported before as far as I remember
However your observations are correct and the code will be worked to correct it
.Pierre
In reply to graham crowder
Re: "Correct" and a red cross can appear simultaneously in a calculated question. Why have it so?
by Pierre Pichet -
The primary objective of calcualted question is to test if the student can solve the correct answer equation within the tolerance and grade him accordingly.
However in technical or physical science, significant figures express more or less the precision of the answer.
Grading the student also on the significant figures used is adding another question on the result i.e. the calculated question should become a multianswer question type and a specific penalty should be applied.
Actual Moodle implementation of calculated is not multianswer, so I propose that we use the exact answer value (without significant figures limit) to test the responses at the two places ( showing the "colored" answer result and in the grading) and add supplementary comment as a feedback in the grading if the student response does not encounter the significant figures limits set by the teacher.
Pierre
P.S. this can be a first step in implementing significant figure grading ...
However in technical or physical science, significant figures express more or less the precision of the answer.
Grading the student also on the significant figures used is adding another question on the result i.e. the calculated question should become a multianswer question type and a specific penalty should be applied.
Actual Moodle implementation of calculated is not multianswer, so I propose that we use the exact answer value (without significant figures limit) to test the responses at the two places ( showing the "colored" answer result and in the grading) and add supplementary comment as a feedback in the grading if the student response does not encounter the significant figures limits set by the teacher.
Pierre
P.S. this can be a first step in implementing significant figure grading ...
In reply to Pierre Pichet
Re: "Correct" and a red cross can appear simultaneously in a calculated question. Why have it so?
by graham crowder -
Pierre,
Thank you for your responses. I agree 100% that the main goal is to check that the student is within the desired tolerance of the exact answer. Consequently, this should be the criterion for the award of Correct/Incorrect and also the Green Tick/Red Cross.
The significant figures can determine the accuracy with which Moodle presents the correct answer but this is not the place to add a judgment about the number of significant figures given by the student.
I woud be delighted to see this impemented. If I can help, let me know though I guess my skills may limit this to helping with any testing.
Graham
Thank you for your responses. I agree 100% that the main goal is to check that the student is within the desired tolerance of the exact answer. Consequently, this should be the criterion for the award of Correct/Incorrect and also the Green Tick/Red Cross.
The significant figures can determine the accuracy with which Moodle presents the correct answer but this is not the place to add a judgment about the number of significant figures given by the student.
I woud be delighted to see this impemented. If I can help, let me know though I guess my skills may limit this to helping with any testing.
Graham
In reply to graham crowder
Re: "Correct" and a red cross can appear simultaneously in a calculated question. Why have it so?
by Pierre Pichet -
Graham,
Thanks for your comments and I will notify you when ready to be tested
Pierre
Thanks for your comments and I will notify you when ready to be tested
Pierre