Grading of multiple choice answers when there is more than one answer

Grading of multiple choice answers when there is more than one answer

by Fiona MacAlister -
Number of replies: 8

My team and I are having problems with the marking of our multiple choice questions that have more than one answer.

The problems are as follows:

If the students tick all of the answers they will automatically get the question right.  This can only be corrected if you insert a minus % for the incorrect answers in which case they will get 0 for the question i.e if there are 6 questions and 3 correct answers, the correct answers are +33% and the incorrect answers are given a value of -33%.  This will also only work for an even number of questions (?).

The example above will also not work for questions where they get two right and one wrong.  They will then only get 1 mark, instead of two because of the -33% for incorrect answers.

If you go back and leave out the negative percentages then they will once again be able to tick every answer and get the question 100% right.  It seems to be a catch 22 situation.

We don't want the students to be able to work in adaptive mode either, because this is a final assessment and not a general quiz where they would be able to check the answer and change it.

Does anyone know a way around this, other than someone having to go through every single quiz afterwards and verify the result of each and every question?  This will be extremely cumbersome, as there are 3 units each with three quizzes in each unit.  Each quiz is, on average, 10 to 15 questions long.  Help!

Average of ratings: -
In reply to Fiona MacAlister

Re: Grading of multiple choice answers when there is more than one answer

by Joseph Rézeau -
Picture of Core developers Picture of Plugin developers Picture of Testers Picture of Translators
Hi Fiona,
In my opinion the scoring system of the multichoice, multiselect question type in Moodle quiz questions is perfectly acceptable. As you have found out, if your question has for example 4 choices, 2 correct and 2 wrong, then correct choices must be given a score of 50% each and wrong choices a score of -50% each... which gives the following scenarios.
Student ticks all four choices: score = 0 (2 x 50% minus 2 x -50%).
Student ticks 2 correct choices: score = 100%
Student ticks 2 wrong choices: score = 0%
Student ticks 1 correct choice plus one wrong choice: score = 0%
Student ticks 2 correct choices plus one wrong choice: score = 50%.
I frankly do not see what else you would consider a "fairer" scoring system.
Joseph
PS.- You might be interested in an excellent explanation of the MULTICHOICE, MULTISELECT scoring logic for the Hot Potatoes software here.
Average of ratings: -
In reply to Joseph Rézeau

Re: Grading of multiple choice answers when there is more than one answer

by Fiona MacAlister -

Thanks for the clarification Joseph.  I see where you're coming from on that - we couldn't see the wood for the trees. mixed  It becomes confusing when you are trying to translate a written multiselect multichoice, where it has previously been marked physically by a person and they are have allocated one mark for one correct answer, to an automated multiselect multichoice. 

The brief we were trying to follow is to allocate one mark to a learner who ticks one correct answer and one incorrect answer.  But I agree with you, if they don't get both right to begin with, they should get no marks for the question, and if they don't read the question correctly and select one more choice than they should then they should be penalised.

Am I correct in thinking that this system will not work with an odd number of choices?

Average of ratings: -
In reply to Fiona MacAlister

Re: Grading of multiple choice answers when there is more than one answer

by Joseph Rézeau -
Picture of Core developers Picture of Plugin developers Picture of Testers Picture of Translators
... with an odd number of choices?
  • For each correct choice: give a score equal to 100 divided by the number of correct choices
  • For each incorrect choice: give a score equal to -100 divided by the number of incorrect choices.

See attached sample.

Joseph

Attachment image00.jpg
Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Grading of multiple choice answers when there is more than one answer

by paramveer william -

Mr. Joseph can you  send code for this demo please !

Average of ratings: -
In reply to paramveer william

Re: Grading of multiple choice answers when there is more than one answer

by AL Rachels -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Hi paramveer william,

Joseph has retired and doesn't visit us very much anymore.

Here is a sample of the question in Joseph's explanation with just one minor difference. In his example, he had the Multiple tries > Penalty for each incorrect try set to 10%. In the sample question I enclosed that penalty is set to the default 33.33333%. The question bank file is in Moodle XML format.


Average of ratings: -
In reply to Joseph Rézeau

Re: Grading of multiple choice answers when there is more than one answer

by Geoffrey Brewster -

"In my opinion the scoring system of the multichoice, multiselect question type in Moodle quiz questions is perfectly acceptable. As you have found out, if your question has for example 4 choices, 2 correct and 2 wrong, then correct choices must be given a score of 50% each and wrong choices a score of -50% each... which gives the following scenarios.
Student ticks all four choices: score = 0 (2 x 50% minus 2 x -50%).
Student ticks 2 correct choices: score = 100%
Student ticks 2 wrong choices: score = 0%
Student ticks 1 correct choice plus one wrong choice: score = 0%
Student ticks 2 correct choices plus one wrong choice: score = 50%.
I frankly do not see what else you would consider a "fairer" scoring system."

Actually, even in this scenario, the grading system is a bit unfair. The student gets the same grade if they get one right/one wrong or if they get two wrong.

A "fairer" grading scheme would be if they got half credit for getting one right regardless of whether they got another part of the question wrong or not.
Average of ratings: -
In reply to Fiona MacAlister

Re: Grading of multiple choice answers when there is more than one answer

by C V -
Similar issue...

This is my question (below), but it doesn't import as it throws an error:

Which of the following files are part of a typical virtual machine? Select 3 response(s). {
~%33%Correct Answer 1
~%33%Correct Answer 2
~%-50%Wrong Answer 1
~%-50%Wrong Answer 2
~%33%Correct Answer 3
}

Anyone see problems with it?

Average of ratings: -