Confidence based marking / Certainty based marking (CBM)

Re: Confidence based marking / Certainty based marking (CBM)

by Joseph Rézeau -
Number of replies: 1
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators
Hi Jeff and Sacha,
Here is my modest contribution to this thread. Using my regular expressions question type in Moodle 1.6, one can do the following.

1- Challenging/commenting questions

Animals
Question text: Name an animal in 3 letters, with vowel a in the middle.
You can add an optional comment (or a question) after your answer, using the ! separator.
Sample answer:
cat !That was too easy.
Answer 1: the [bcr]at
Feedback:
Correct Grade: 100%
Answer 2: the [bcr]at\s?!.*
Feedback:
Correct. Thanks for your comment. Grade: 100%
Answer 3: --the.*
Feedback:
Missing definite article! Grade: 0%
Answer 4: .*!.*
Feedback: Your answer is probably wrong, but thanks for your comment. Grade: 0%

Sample student answers:

Student answer #1: the rat !that was dead easy...
Feedback: Correct. Thanks for your comment.
Student answer #2: dunno
Feedback: Missing definite article.

2- Confidence indicator

Animals (extinct)
Question text: Which (one) of these animals is extinct? the cat, the mammoth, the tiger, the elephant. Please add an indicator of your confidence after your answer. From C1 (not sure); C2 (maybe); C3 (absolutely certain).
Sample answer: the cat C1

Answer 1: the mammoth\s*[cC]3
Feedback: Correct. Indeed, the mammoth is quite extinct. Grade: 100%
Answer 2:
--.*[cC][123]
Feedback:
You forgot the Confidence indicator. Grade: 0%
Answer 3:
the mammoth\s*[cC]1
Feedback:
Why do you doubt that the mammoth is extinct? Grade: 33.33%
Answer 4:
the mammoth\s*[cC]2
Feedback: Why are you not quite sure that the mammoth is extinct? Grade: 66.66%
Answer 5:
.*\b(cat|tiger|elephant)\b.*
Feedback: That's not an extinct animal. Grade: 0%

Actually we should be able to give negative scores for wrong student answers associated with a high confidence indicator. e.g.

.*\b(cat|tiger|elephant)\b\s*[cC]3 -> Grade -66%

.*\b(cat|tiger|elephant)\b\s*[cC]2 -> Grade -33%

.*\b(cat|tiger|elephant)\b\s*[cC]1 -> Grade 0%

Unfortunately questions cannot have a negative score in Moodle at present, so the negative scale of scores is not available in SHORTANSWER not in REGEXP question types.

Use at own risk.black eye The attached screen shot shows how, with careful use of regexp expressions to analyse the student's answers and provide meaningful (automatic) feeedback, plus manual, human comment by teacher (plus student's challenge/comment facility), a kind of pedagogic "pseudo-dialogue" can be established.

Joseph

Attachment image-0000.jpg
Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Confidence based marking / Certainty based marking (CBM)

by Sacha Brostoff -

Hi Jef and Joseph!

Thank you both (and Joseph for giving me a such a head-start on the regexp coding!) - this is all looking *very* promising.

I hope to have a working Moodle server next week, so I can start experimenting.  I may well have lots of questions for you!

Best regards,

Sacha