Numerical questions - invalid answers

Numerical questions - invalid answers

by Henning Bostelmann -
Number of replies: 11
Picture of Core developers Picture of Plugin developers

Hello,

Numerical questions currently have the following behaviour: If someone enters an answer that is not a number, the answer is graded as incorrect.

Thus, for example, if the correct answer is "0.25" and the student enters "1/4", or if the correct answer is "2" and the student enters "two", this is counted as incorrect, and a penalty applied.

On the other hand, if a teacher tries to enter "two" as an (incorrect) answer in the question editing screen, in order to provide feedback, this input is rejected by the form validation code: "The answer must be a number."

I was discussing with Gustav Delius and we weren't sure whether this is a bug or a feature. (An alternative behaviour would be to reject non-numeric answers without penalty.)

Can anyone shed light on this issue?

Henning

Average of ratings: -
In reply to Henning Bostelmann

Re: Numerical questions - invalid answers

by Pierre Pichet -

This is more a feature than a bug.

If you want to authorize various numerical formats without any requirements about tolerance then you shoud use shortanswer type.

The shortanswer will accept any text or number as a valid response even 1/4 .

I use shortanswer in 1,9 as I want that the student use 3,1416 instead of 3.1416 as , is the decimal separator in french.

Pierre

P.S. "discussing with Gustav Delius" 

Happy to have some indirect news of Gustav who was the Quiz module leader before Tim,

 

Average of ratings: -
In reply to Henning Bostelmann

Re: Numerical questions - invalid answers

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

The new question engine is built with a concept of 'validation' included, and indeed, I think we already do some validation of numerical answers. I am certainly receptive to the idea of making the validation better, if you want to give it a go, but I have to say, "Here be dragons".

As Pierre hinted, we have to work in all locales, which means that 1,000 is ambiguous.

We also have to cope with both £100 and 100m, and we have to cope with numbers like 3.0x10^8, and 3e8, but also units like m3.

We don't yet try to do accountants' negatives that look like (£1000) for what we (I think we are all scientists) would write -£1000. Looking in our database, I have seen students on accounting courses write that, and get marked wrong.

So, you are welcome to try to make things better, but please provide copious (extra) unit tests with your code to prove it does what you claim, and good luck, because I gave up where we currently are, even though I know it is not quite good enough. Pierre is a really helpful source of advice about the , issue, being a French Canadian.

Average of ratings: -
In reply to Tim Hunt

Re: Numerical questions - invalid answers

by Henning Bostelmann -
Picture of Core developers Picture of Plugin developers

Hi Tim,

yes, I might look into that at some point. Just for clarification: I was not proposing that the numerical question should recognize more formats. I just suggested that, if an input is not recognized as a valid number, the response should be disregarded without penalty. Ideally this validation would use the same code as the current one used for parsing the answer, with a few modifications. But honestly I haven't looked into details yet.

Average of ratings: -
In reply to Henning Bostelmann

Re: Numerical questions - invalid answers

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I think it does use the same code, but I have not reviewed that bit of code for a while.

Average of ratings: -
In reply to Henning Bostelmann

Re: Numerical questions - invalid answers

by Pierre Pichet -

"I just suggested that, if an input is not recognized as a valid number, the response should be disregarded without penalty."

This is somehow a new feature that will need a more specific proposal given all the options of the new engine grading.

First what could be the non valid response without a penalty ?

 

Pierre

Average of ratings: -
In reply to Pierre Pichet

Re: Numerical questions - invalid answers

by Henning Bostelmann -
Picture of Core developers Picture of Plugin developers

> First what could be the non valid response without a penalty ?

Any answer that is not a number (or, more precisely, not in the expected number format). For example, if the student types "1/10" instead of "0.1".

At least in the context where we use quizzes (or in HE in general), we can expect that students know in principle what a number is. Thus, if the question asks for something numeric and the student's response is not parseable to a number, then in almost all cases that is for purely formal reasons (a misunderstanding in notation). In this case, a penalty is hardly justified. Instead, ignoring the response and displaying a help text about allowable number formats seems more appropriate.

Funny enough, I just realized that numerical questions do behave like this in some situations. For example, this seems to work as just described in "interactive" mode. In "adaptive" mode, which I tested first, it doesn't. So, there seems to be a bug around.

Average of ratings: -
In reply to Henning Bostelmann

Re: Numerical questions - invalid answers

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Ah, I have probably never got around to thinking through how adaptive behaviour should work in relation to is_valid. If you are able to give that some thought, that would be great.

Average of ratings: -
In reply to Tim Hunt

Re: Numerical questions - invalid answers

by Henning Bostelmann -
Picture of Core developers Picture of Plugin developers

MDL-30031. I won't get to it during the next few weeks, though.

Interestingly, I have a custom question type here for which adaptive mode does ignore invalid answers. Maybe it's a confusion on my side about the API.

Average of ratings: -
In reply to Henning Bostelmann

Re: Numerical questions - invalid answers

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

I use javascript code to interpret the textbox response once entered, replacing it with a standard numerical format (if it isn't already OK) and alerting the student if changes have been made or if there is a problem to check it or put it into a valid numerical format before submission.

 

Average of ratings: -
In reply to Tony Gardner-Medwin

Re: Numerical questions - invalid answers

by Pierre Pichet -

I think that something like the hint feature of the new question engine will do the job more inline with moodle handling of quiz and questions.

But one on the beauty of open code is that you can tailor it to your specific needs smile.

Pierre

Average of ratings: -
In reply to Henning Bostelmann

Re: Numerical questions - invalid answers

by Sharon Strauss -
Picture of Testers

Hi,


I agree with Tim.  I got a question from a student who submitted this quiz answer:

bad error report warningIn this case, the problem was the upper case "X"

If the error was worded as something like, "Moodle does not recognize at least one of the numbers you entered" the student probably would have figured out her problem. Instead, she was confused and contacted her instructor--who then contacted me.


Average of ratings: -