As a moodle newbie, I'm struggling to find an effective solution to the following problem.
I have a question which has multiple choice answers, and has more than 1 correct answer. I can't find a way to grade it so that a user gets 1 point for a correct answer, but gets penalised for additional wrong answers. Otherwise the canny user could select all possible answers, and get full marks.
Assume the correct answers are A and B out of a choice of A, B, C and D.
I would like to set it up the following way:
- If the user selects A and B, he gets 2 points.
- If the user selects A, he gets 1 point.
- If the user selects A, B and C, he gets only 1 point (2 points for the correct answer, but -1 for the wrong answer).
All these I can manage by setting A and B to 50% each, and C and D to -50% each.
But if the user selects A and C, he should get 1 point, because he got A right, whereas he'll get 0 points (+50% for A and -50% for C). That doesn't seem right. He should only get penalised if he gets ADDITIONAL wrong answers in addition to the correct ones.
It seems a fairly obvious way to score things, but there doesn't seem an obvious solution. Does anyone have any workarounds? So far we've done it by using the Matching type of question and setting each possible answer to True or False, but it's not quite the same and is very clunky.
In reply to Diana Maynard
Re: grading multiple choice questions with multiple answers
by Joseph Rézeau -
Hi Diana,
There have already been countless discussions in this forum about the topic of scoring "multiple choice, multiple answers" questions. You say "It seems a fairly obvious way to score things, but there doesn't seem an obvious solution." In fact things are not so "obvious" because different people have different views on the problem and it is nearly impossible to achieve a consensus.
The settings you are using are correct (setting A and B to 50% each, and C and D to -50% each), but then I disagree with your reasoning when you say that "if the user selects A and C, he should get 1 point". That reasoning totally contradicts your earlier reasoning that "the canny user could select all possible answers, and get full marks".
How would you score the following:
- student S1 selects answers A (correct) and C (incorrect)
- student S2 selects answer A (correct)
The settings you are using (and which are possible in Moodle) would score student S1 = 0 points and student S2 = 1 point.
From your reasoning above, I expect you would want to score both students 1 point. This is what you call "a fairly obvious way to score things". To me it is unfair.
Joseph
PS.- My only hope in posting this is to make people aware that there are different ways to see this problem not just one "obvious" way, and that we shall continue to "agree to disagree".
In reply to Joseph Rézeau
Re: grading multiple choice questions with multiple answers
by Diana Maynard -
I should have mentioned a couple of things.
(1) I did try about a dozen searches in the forum to see if this topic had previously been mentioned, but found nothing. Obviously i didn't get the combination of keywords exactly right. I couldn't see any obvious classification of topics in this module to search further.
(2) When I said "it's an obvious way" I didn't mean it was the only way. I just mean it's a method that I suspect many more people than just me want to do. Independently in our group, all 3 of us wanted to do exactly the same thing (setting different questions, and having made them up from scratch).
The rationale is that you get a point for every correct answer (seems normal) and you don't get a point for an incorrect answer (also seems normal), but at the same time there should be a way to prevent someone just selecting every possible answer and getting full marks. I think it's obvious that the latter is plain wrong.
I appreciate that it's hard to factor this into the design of the system, but my question was "has anyone got a suitable workaround to this issue?"
Diana
(1) I did try about a dozen searches in the forum to see if this topic had previously been mentioned, but found nothing. Obviously i didn't get the combination of keywords exactly right. I couldn't see any obvious classification of topics in this module to search further.
(2) When I said "it's an obvious way" I didn't mean it was the only way. I just mean it's a method that I suspect many more people than just me want to do. Independently in our group, all 3 of us wanted to do exactly the same thing (setting different questions, and having made them up from scratch).
The rationale is that you get a point for every correct answer (seems normal) and you don't get a point for an incorrect answer (also seems normal), but at the same time there should be a way to prevent someone just selecting every possible answer and getting full marks. I think it's obvious that the latter is plain wrong.
I appreciate that it's hard to factor this into the design of the system, but my question was "has anyone got a suitable workaround to this issue?"
Diana
In reply to Diana Maynard
Re: grading multiple choice questions with multiple answers
by Daniel McCloy -
One way we've approached this in my dept. is to make A & B worth +50% and make C & D worth -25%. It still penalizes someone who just selects all the answers --- just that they get 50% instead of zero. And it still gives credit to someone who selects one right and one wrong answer --- just that they get 25% instead of 50%.
I admit that it's not ideal, because someone who made a genuine effort but guessed wrong on one answer could get 25% while someone just trying to game the system by selecting all answers could do better. But regardless, gaming the system only nets 50% which is not a passing grade.
Note also that such questions are really problematic in adaptive mode: a student can select all checkboxes, submit the question, immediately know which answers were correct, then resubmit only the correct answers for a 90% grade (assuming the default 10% penalty). Someone who didn't know the answer but was "playing fair" might take 2 or 3 iterations of adaptive mode to hit on the correct answers, and thus score 80 or 70%. Again, not ideal.
I admit that it's not ideal, because someone who made a genuine effort but guessed wrong on one answer could get 25% while someone just trying to game the system by selecting all answers could do better. But regardless, gaming the system only nets 50% which is not a passing grade.
Note also that such questions are really problematic in adaptive mode: a student can select all checkboxes, submit the question, immediately know which answers were correct, then resubmit only the correct answers for a 90% grade (assuming the default 10% penalty). Someone who didn't know the answer but was "playing fair" might take 2 or 3 iterations of adaptive mode to hit on the correct answers, and thus score 80 or 70%. Again, not ideal.
In reply to Daniel McCloy
Re: grading multiple choice questions with multiple answers
by Diana Maynard -
Thanks, that's really helpful!
In reply to Diana Maynard
Re: grading multiple choice questions with multiple answers
by Megan McCarron -
It seems to me that the most ideal solution would be the ability to provide points for leaving incorrect responses unselected. This is the way I would score this type of question if I were scoring it by hand. This way each response receives the same point value and you don't run into the issue of total points exceeding 100% per question nor do you have to award negative point values for any question. By setting the system up to award negative values, you run into the potential issue of a student earning an overall negative point value for the question (if they select more incorrect options than they do correct ones).
So in the example of 5 options 3 of which are correct (A,B and D), 2 of which are incorrect (C and E). Each response would be assigned a value of 20%. So a student would be awarded points for selecting all correct responses and likewise, get awarded points for leaving incorrect responses unselected.
Selecting A = 20% (0% awarded if this response is not selected)
Selecting B = 20% (0% awarded if this response is not selected)
Leaving C unselected = 20% (0% awarded if this response is selected)
Selecting D = 20% (0% awarded if this is not selected)
Leaving E unselected = 20% (0% awarded if this is selected)
I'm sorry if this suggestion has already been discussed elsewhere.
Megan
So in the example of 5 options 3 of which are correct (A,B and D), 2 of which are incorrect (C and E). Each response would be assigned a value of 20%. So a student would be awarded points for selecting all correct responses and likewise, get awarded points for leaving incorrect responses unselected.
Selecting A = 20% (0% awarded if this response is not selected)
Selecting B = 20% (0% awarded if this response is not selected)
Leaving C unselected = 20% (0% awarded if this response is selected)
Selecting D = 20% (0% awarded if this is not selected)
Leaving E unselected = 20% (0% awarded if this is selected)
I'm sorry if this suggestion has already been discussed elsewhere.
Megan
In reply to Megan McCarron
Re: grading multiple choice questions with multiple answers
by Sarra Lev -
This is so smart, but is there a way to do this in moodle?!
You could do it as a matching question, where each subquestion has answer Yes or No.
Can you explain a little more, or send me to where I should go to learn more? The way I've done matching questions, you could only have one to one. If there are several of the same answers on one "side" (yes or no), how would the program know which "yes" or "no" belongs to which question on the other side of the match?
Does this make any sense?
The program is clever. Try it, and you will find that it works.
I'm sorry. I cannot figure out how to do this.
In a multiple choice with multiple answers, I have a question and then each choice goes in "choice #1," "choice #2," etc. along with either a positive or negative grade.
For example, my ideal would be:
Question: Canada is:
Choice #1: North of the US - (20%)
Choice #2: A country - (20%)
Choice #3: Cold all year-round - (20% if they don't choose this)
Choice #4: More populated than the US - (20% if they don't choose this)
Choice #5: A monarchy - (20% if they don't choose this)
Etc.
If I do it the way you suggest, in a "matching" question, where do I put the words "Canada is" and then where do I put the choices and the Yes/No?
Sorry to be so dense.
SL
I believe the suggestion is to use Embedded Answers (Cloze) question type. The syntax for your example would be
Question: Canada is:Choice #1: North of the US - {1:MULTIPLECHOICE:%100%Y~%0%N}
Choice #2: A country - {1:MULTIPLECHOICE:%100%Y~%0%N}
Choice #3: Cold all year-round - {1:MULTIPLECHOICE:%0%Y~%100%N}
Choice #4: More populated than the US - {1:MULTIPLECHOICE:%0%Y~%100%N}
Choice #5: A monarchy - {1:MULTIPLECHOICE:%0%Y~%100%N}
This question by default 5 points, but can be scaled in the question editor.
I'd like to revisit this, if possible. I did try a few of these suggestions, but found that each is problematic in its own way:
1. Embedded Answers (Cloze) question type - the answers cannot be randomized internal to the question as I generally do. This is necessary because I have them repeat the question throughout the semester and don't want them to simply memorize the correct answers by placement.
2. True/False - this works, but is very cumbersome as each answer requires an entirely new question.
3. Matching - (Tim's suggestion) I never did figure out how to make it work with this.
4. "you could present the options ABCD and have them chose ONE of the combinations" (Jeff's suggestion) leaves open the option of too many opportunities for "stupid" mistakes.
I'm wondering if there is anyone else who would like the functionality of grading students for what they do not select, in addition to what they do select, and if I might put in a plea for it (and where do I do this?). It seems the best scoring system I can come up with for my needs and is certainly the least cumbersome for them.
Thanks so much.
SL
There is a new feature coming in Moodle 3.2 that allows adding multiple response questions in embedded answer questions with or without randomization. See previous discussion https://moodle.org/mod/forum/discuss.php?d=335474
Thank you!
Hi Sarra,
Just a quick note to point out that "1. Embedded Answers (Cloze) question type - the answers cannot be randomized internal to the question as I generally do.", did not work for you probably due to not adding _S to define the answers as needing to be shuffled.
Here is a sample:
This is a sample CLOZE Multiple Choice Shuffled type question. {1:MULTICHOICE_S:~%100%Pick me to be correct#correct~%0%Pick me to be wrong 1#incorrect~%0%Pick me to be wrong 2#incorrect~%0%Pick me to be wrong 3#incorrect}
Try it and you will see that the answers do get shuffled.
Thank you. I'm actually trying to achieve two different things that it thus far seems can't be mixed. The first, is to assign (positive) scores to options that the students DON'T choose just as there are scores for those they DO choose. To do that, someone suggested I use the matching question in CLOZE, and give a yes/no match. That's worked, for that purpose, but can't be randomized.
The multiple choice, on the other hand, can't score the way I'd like it to. I don't like the negative numbers. It never works pedagogically for me.
Does that help explain better?
Thanks Al.
S
In reply to Sarra Lev
Re: grading multiple choice questions with multiple answers
by Antonia Bonaccorso -
Hi Sarra
There
is a question type called kprime
(https://moodle.org/plugins/qtype_kprime) that offers the possibility to
have a stem (question) and four options. For each of the options the
student has to choose whether this option is right or wrong. (Remark:
Right/wrong can be replaced by any dichotomous words or categories like
black/white)
There are three scoring methods available
- Subpoints: The student receives 25% of the total points for each correctly answered option.
- KPrime:
The student receives 100% of the total points if all four options are
answered correctly, 50% of the total points if three options are
answered correctly and 0% if less than 3 options are answered correctly.
- KPrime 1/0: The student receives 100% of the total points if all options are answered corretly and 0 point otherwise.
- each option has to be clearly right or wrong
- you must enter four options
Best regards,
Antonia
In reply to Daniel Thies
Re: grading multiple choice questions with multiple answers
by William Mawunyo Agbo -
Daniel, will the syntax below as you wrote give an output similar to the image I have attached?
Question: Canada is:
Choice #1: North of the US - {1:MULTIPLECHOICE:%100%Y~%0%N}
Choice #2: A country - {1:MULTIPLECHOICE:%100%Y~%0%N}
Choice #3: Cold all year-round - {1:MULTIPLECHOICE:%0%Y~%100%N}
Choice #4: More populated than the US - {1:MULTIPLECHOICE:%0%Y~%100%N}
Choice #5: A monarchy - {1:MULTIPLECHOICE:%0%Y~%100%N}
I am trying to achieve exactly as in image attached.
In reply to Diana Maynard
Re: grading multiple choice questions with multiple answers
by Joseph Rézeau -
Hi Diana,
(1) A forum search for "score multiple answers" forum : quiz gives 12 pages of results.
(2) Point taken. However, 3 persons out of 3 sharing the same view hardly counts from a statistical point of view. It would be interesting to conduct a survey among moodle quiz users to find out more.
(3) I quite understand you rationale. However, other quiz users may argue that an incorrectly checked answer is not simply worth "no points", it should be penalized (i.e. worth -1).
I hope you'll find Daniel's suggestion useful.
Oh, and, doing that forum search I gave you the keywords for, I came upon one of my earlier answers on this matter, with a reference to Hot Potatoes, here.
ATB
Joseph
In reply to Joseph Rézeau
Re: grading multiple choice questions with multiple answers
by Velson Horie -
Yet another twist, which I haven't seen covered.
I wish to set a multiple choice question with 5 responses, 3 of which are correct and 2 incorrect. I am asking for 2 responses in order to get 100%.
So I graded each of the three correct responses at 50% and the rest at -50%. But Moodle will not accept the question with the potential total mark more than 100%, because the number of responses cannot be limited. Moodle will facilitate a question with a potential negative score but award it 0%. Can Moodle cope with a question with a potential score >100% but award it a maximum of 100%?
It would be useful to be able to specify the minimum and maximum number of responses needed for a valid submission.
Velson
I wish to set a multiple choice question with 5 responses, 3 of which are correct and 2 incorrect. I am asking for 2 responses in order to get 100%.
So I graded each of the three correct responses at 50% and the rest at -50%. But Moodle will not accept the question with the potential total mark more than 100%, because the number of responses cannot be limited. Moodle will facilitate a question with a potential negative score but award it 0%. Can Moodle cope with a question with a potential score >100% but award it a maximum of 100%?
It would be useful to be able to specify the minimum and maximum number of responses needed for a valid submission.
Velson
In reply to Joseph Rézeau
Re: grading multiple choice questions with multiple answers
by Josh Lange -
Hi Joseph!
Very competent and helpful reply! You seem to really know your stuff. I'm encountering a different problem and you seem to have the deep thinking to solve it quite easily, if you don't mind helping?
The institution offers an offline-online split assessment system where Moodle is used for a pre-determined set of multiple choice types. They never want negative answers, only a return of =0 for wrong answers.
The complexity comes in now. There are a maximum of four choices and potential correct answers that could be N = one, two, three, or four correct answers. The correct answer(s) are 100%/N = (% of one point) but if a correct answer is not chosen, the test taker does not get any score.
Do you know which options I have for this in Moodle? I am using a German language macro that offers to rate "false answers" in 3 ways! Thanks so much, awesome posts!
Josh
In reply to Diana Maynard
Re: grading multiple choice questions with multiple answers
by Jeff Forssell -
If it is REALLY that important to get that grading scheme (or an even more refined grading) than you could present the options ABCD and have them chose ONE of the combinations below (I hope I haven't missed any combinations! 
)
1) A; 2) B; 3) C; 4) D
5) AB; 6) AC; 7) AD; 8) BC; 9) BD; 10) CD
11) ABC; 12) ABD; 13) ACD; 14) BCD
15) ABCD
)1) A; 2) B; 3) C; 4) D
5) AB; 6) AC; 7) AD; 8) BC; 9) BD; 10) CD
11) ABC; 12) ABD; 13) ACD; 14) BCD
15) ABCD