Hi...
I am using moodle 1.8, stable version. The Quiz module has a defined system for negatives.
What i require is this.
Assume a multiple choice question (MCQ) with 4 options.
If the student clicks on the correct answer, he gets +3. plus if he clicks on the wrong answer, he gets -1 marks. also he can attempt the question only once. not attempt it, submit n realize it is wrong and then answer again and again till he gets the right answer.at the end of the quiz, the evaluation assigns marks calculating all the +3 for right answer, -1 for wrong answer and 0 for no attempt.
problems:
1. adaptive mode has to be on for the negatives. if kept so, the negatives will be assigned only on repetitive answering of the wrong answer.
2. if the student answers a wrong answer, but does not click there to submit n check, at the end of the quiz, those negatives do not get subtracted. it says he got 0/3 for that question. it should be -1.
3. any way by which the option of negatives can hold, but the submit button after each question can be removed. i.e only one submit button after the test.?...this can be done by keeping adaptive mode off, but then can negatives still be assigned?
I would greatly appreciate any help regarding this topic. and i really need to get such a functionality working, so i hope the moodle community can help me here in some way.
Thanking you all in anticipation,
Yatin
Goa-India.
Unfortunately, the minimum grade that a student can receive for a question is zero. Negative grades for questions is a frequently requested feature. Vote for MDL-1647, which has been open since Moodle 1.4.5. Also see this discussion and others like it.
In reply to John Isner
Re: Need Urgent Help! problem with negatives in quiz module
by Joseph Rézeau -
John > Unfortunately, the minimum grade that a student can receive for a question is zero.
Fortunately, the minimum grade that a student can receive for a question is zero.
I am sincerely astounded at the idea of grading a student less than zero. Maybe as a non-maths/science teacher the very idea of negative aptitude is beyond my comprehending.
Joseph
In reply to Joseph Rézeau
Re: Need Urgent Help! problem with negatives in quiz module
by Steve Hyndman -
Maybe as a non-maths/science teacher the very idea of negative aptitude is beyond my comprehending.
Here is the argument...or at lest one argument, for negative scoring.
Say you administer to a group of students a 100 question, multiple choice test, with each question having 4 possible responses...of course only one of the 4 responses would be "correct" for each question.
If a student didn't read any of the questions and simply filled in the answers, what grade would you expect them to get?
You may initially think 0, but of course, that wouldn't be correct. Random variation would result in the student, on average, getting a 25. So that begs the question; How can a student have an "aptitude" of 25 when they didn't even read the questions? Answer...probability and statistics.
I taught stats for years and drove this point home with my students with a simple example. I would always tell them they could take the final stats exam as a 100 point multiple choice test, and they could choose to take it in one of two ways: 1) the "traditional" way, where they got the percent correct...i.e., if they miss 20, then they get 80%, or, 2) the "very brave" way, where I would give them 100% if they got every question wrong, but if they got one or more questions correct, then I would give them 0%. After they thought about that for a minute, most understood why a 0 on an objective test doesn't mean absence of aptitude
One answer to this problem in objective type testing is negative marking as explained pretty well here: http://web.uct.ac.za/projects/cbe/mcqman/mcqchp4.html
I seem to remember, Dantes test using this method years ago (+ some value for correct response, 0 for no answer, and - some value for incorrect response).
Using my example above, one could argue (and prove mathematically) that a student would have to be pretty smart (on the subject they are being tested on) to get a 0
Steve
In reply to Steve Hyndman
Re: Need Urgent Help! problem with negatives in quiz module
by Greg Johnson -
Multiple choice questions on the US College Board Advanced Placement exams get negative scores to discourage guessing. These are aggregated through a complex calculation to a score 1,2,3,4, or 5.
In reply to Greg Johnson
Re: Need Urgent Help! problem with negatives in quiz module
by Steve Hyndman -
''...get negative scores to discourage guessing."
Yes and there is a lot of validity to that approach. The idea is that if a student doesn't know and doesn't guess then that solves the random variation problem I illustrated above, i.e., example of 25% due to chance from just fulling in answers. Theoretically, if a student only answers question they know, then they get a true score...if they know them all and answer them all, then they get 100%...if they know none of them and don't attempt any of them, then they get 0%...change doesn't come into play.
Steve
In reply to Joseph Rézeau
Re: Need Urgent Help! problem with negatives in quiz module
by John Isner -
Hi Joseph,
Besides the statistical argument, which Steve has presented very nicely, there is a practical reason for negative scores on questions. When we (in the US, and now I see India!) use Moodle to prepare students for standardized tests, we must use the same methodology in grading our practice tests as the standardized tests themselves. Otherwise, students' scores on the practice tests will not predict how students will do on the actual tests. In the US, most students who plan to go to college take either the SAT or the ACT tests. These deduct 0.25 of the point value of every incorrectly-answered multiple choice question. You may have a negative grade for an individual question, but not an overall negative gradefor the test, since you start off with 200 points. Because Moodle does not have negative marks for questions, I teachers must grade such practice tests by hand. It takes 3-5 minutes PER TEST to do the necessary calculations. I would rather that Moodle did it for me, automatically.
Besides the statistical argument, which Steve has presented very nicely, there is a practical reason for negative scores on questions. When we (in the US, and now I see India!) use Moodle to prepare students for standardized tests, we must use the same methodology in grading our practice tests as the standardized tests themselves. Otherwise, students' scores on the practice tests will not predict how students will do on the actual tests. In the US, most students who plan to go to college take either the SAT or the ACT tests. These deduct 0.25 of the point value of every incorrectly-answered multiple choice question. You may have a negative grade for an individual question, but not an overall negative gradefor the test, since you start off with 200 points. Because Moodle does not have negative marks for questions, I teachers must grade such practice tests by hand. It takes 3-5 minutes PER TEST to do the necessary calculations. I would rather that Moodle did it for me, automatically.
Isn't auto-linking great? Just in case you were wondering, there is a 3rd party plugin called p-r-a-c-t-i-c-a-l, and we all need to know that s-t-a-n-d-a-r-d-i-z-e-d is a synonym for m-a-r-t-i-n-i-z-e-d, a dry cleaning process used in the United States. 
In reply to John Isner
Re: Need Urgent Help! problem with negatives in quiz module
by Steve Hyndman -
Yes, so be cafeful of your "choice" of words because we all know that is an activity module
In reply to Yatin Kamat
Re: Need Urgent Help! problem with negatives in quiz module
by Pierre Pichet -
All standard evaluation textbook take for granted that we use a 0-100% score scale.
Some non-standard pratice try to avoid cheating or guessing by using negative score.
The really good way to do it is to propose to the student a so numerous choice that guessing will not influence really the final grade. (at extreme solution suppose a 100 choices for every question...
For practical reason (it is difficult to find 10 or 100 plausible false responses) they forget about theory and set a 4:1 ratio.
That is to say that 1 good answer will be nullify by 4 bad answers so the students will not try guessing.
Moodle want to stay to good theoretical ground in defining his quiz.
This illustrate the effect of the 1:4 ratio i.e 1 good is negated by 4 bad ones.
This is a distorded scale
10 good responses give you 100%
9 good responses give you 87.5%
8 good responses give you 7.5%
and so on.
This is why they use a 100 questions test so that the distortion is less apparent.
100 good responses give you 100%
99 good responses give you 98.75%
In either case 2 good responses/10 gives you 0%
or 20 good responses/100 gives you 0% i.e 20 - (80 * 0,25) = 0
So the final zero is equivalent to answering correctly 20% of the questions.
If you want that your students understand how they will perform in a administrative standard scale, use the general feedback with a reference to a table that will show them how to convert there Moodle results to the administrative score.
Moodle will maintain its standards and your students will learn something about "real" life and equity.
Pierre
Some non-standard pratice try to avoid cheating or guessing by using negative score.
The really good way to do it is to propose to the student a so numerous choice that guessing will not influence really the final grade. (at extreme solution suppose a 100 choices for every question...
For practical reason (it is difficult to find 10 or 100 plausible false responses) they forget about theory and set a 4:1 ratio.
That is to say that 1 good answer will be nullify by 4 bad answers so the students will not try guessing.
Moodle want to stay to good theoretical ground in defining his quiz.
This illustrate the effect of the 1:4 ratio i.e 1 good is negated by 4 bad ones.
| q # |
stud 1 |
stud2 |
stud 3 |
stud 4 |
stud 4 |
| 1 |
1 |
1 |
1 |
1 |
1 |
| 2 |
1 |
-0.25 |
1 |
1 |
1 |
| 3 |
1 |
-0.25 | 1 |
1 |
1 |
| 4 |
1 |
-0.25 | 1 |
1 |
1 |
| 5 |
1 |
-0.25 | 1 |
1 |
1 |
| 6 |
1 |
1 |
-0.25 | 1 |
1 |
| 7 |
1 |
-0.25 | -0.25 | 1 |
1 |
| 8 |
1 |
-0.25 | -0.25 | 1 |
1 |
| 9 |
1 |
-0.25 | -0.25 | 1 |
0 |
| 10 |
1 |
-0.25 | 1 |
-0.25 | 0 |
| 10 |
0 |
5 |
8.75 |
7.5 |
This is a distorded scale
10 good responses give you 100%
9 good responses give you 87.5%
8 good responses give you 7.5%
and so on.
This is why they use a 100 questions test so that the distortion is less apparent.
100 good responses give you 100%
99 good responses give you 98.75%
In either case 2 good responses/10 gives you 0%
or 20 good responses/100 gives you 0% i.e 20 - (80 * 0,25) = 0
So the final zero is equivalent to answering correctly 20% of the questions.
If you want that your students understand how they will perform in a administrative standard scale, use the general feedback with a reference to a table that will show them how to convert there Moodle results to the administrative score.
Moodle will maintain its standards and your students will learn something about "real" life and equity.
Pierre
In reply to Pierre Pichet
Re: Need Urgent Help! problem with negatives in quiz module
by John Isner -
I think there's a mistake in the total for stud 4.
I don't see any distortion in these results. Stud 4 doesn't know the answer to two questions, and his score is the same as it would be on a test without negative scores. Stud 3 knows nine answers. On a test without negative scores, he would get 9.0, but he loses a quarter point for a wrong answer. If he was unsure of his answer, he should have left it out and received 9.0. But he knew the rules (penalty for wrong answer), so we must assume that he thinks his (wrong) answer is correct, and we must penalize him for that!
I don't think it matters if the test has ten questions, a hundred questions, or a thousand questions. If the student gets the same proportion of questions right, wrong, and omitted, he will get the same score as in the above table after scaling the raw score to a max quiz grade of 10.
I do believe that a score based on a larger test is statistically more significant, but that's a different issue.
| q # |
stud 1 |
stud2 |
stud 3 |
stud 4 |
stud 4 |
| 1 |
1 |
1 |
1 |
1 |
1 |
| 2 |
1 |
-0.25 |
1 |
1 |
1 |
| 3 |
1 |
-0.25 | 1 |
1 |
1 |
| 4 |
1 |
-0.25 | 1 |
1 |
1 |
| 5 |
1 |
-0.25 | 1 |
1 |
1 |
| 6 |
1 |
1 |
-0.25 | 1 |
1 |
| 7 |
1 |
-0.25 | -0.25 | 1 |
1 |
| 8 |
1 |
-0.25 | -0.25 | 1 |
1 |
| 9 |
1 |
-0.25 | -0.25 | 1 |
0 |
| 10 |
1 |
-0.25 | 1 |
-0.25 | 0 |
| 10 |
0 |
5 |
8.75 |
8.0 |
I don't see any distortion in these results. Stud 4 doesn't know the answer to two questions, and his score is the same as it would be on a test without negative scores. Stud 3 knows nine answers. On a test without negative scores, he would get 9.0, but he loses a quarter point for a wrong answer. If he was unsure of his answer, he should have left it out and received 9.0. But he knew the rules (penalty for wrong answer), so we must assume that he thinks his (wrong) answer is correct, and we must penalize him for that!
I don't think it matters if the test has ten questions, a hundred questions, or a thousand questions. If the student gets the same proportion of questions right, wrong, and omitted, he will get the same score as in the above table after scaling the raw score to a max quiz grade of 10.
I do believe that a score based on a larger test is statistically more significant, but that's a different issue.
In reply to John Isner
Re: Need Urgent Help! problem with negatives in quiz module
by Pierre Pichet -
Your rigth, I did not understand that what you are trying to grade, is that the student knowledge is 100% about a specific question. So if he doesn't know the answer, he accepts the 0 grading and doesn't try anything else.
Your understanding is that trying something when you are not 100% sure, is cheating.
Your questions need to be thorougly tested that there no bias or ambiguity in the 3 false answers. This is rarely the case in real life.
You say that these are official tests to which they must comply.
However, we have the responsability that the grading process use in Moodle regular quiz reflects normal grading practice as these grades are the official ones in the college or university student's record.
If there is a real need of these special grading rules, I will prefer that we create a specific new quiz type that will use only a special multichoice question type ( is there other questiontypes used?) so that the two grading process are not mixed or used by inadvertance if the teacher click the bad choice.
Pierre
Your understanding is that trying something when you are not 100% sure, is cheating.
Your questions need to be thorougly tested that there no bias or ambiguity in the 3 false answers. This is rarely the case in real life.
You say that these are official tests to which they must comply.
However, we have the responsability that the grading process use in Moodle regular quiz reflects normal grading practice as these grades are the official ones in the college or university student's record.
If there is a real need of these special grading rules, I will prefer that we create a specific new quiz type that will use only a special multichoice question type ( is there other questiontypes used?) so that the two grading process are not mixed or used by inadvertance if the teacher click the bad choice.
Pierre
In reply to Pierre Pichet
Re: Need Urgent Help! problem with negatives in quiz module
by John Isner -
Hi Pierre,
I know that on the SAT test, the questions themselves are tested on hundreds of thousands of students before they are actually counted. I'm sure that rigorous item analysis is performed.
I don't really see these as "special grading rules." We already have a penalty factor in adaptive mode. This is also a penalty factor, except that it happens to exceed 1.0 (it would be 1.25 for SAT). I'm not a developer, so I don't understand the coding implications of such a change, but it does seem like a natural extension of the penalty factor. To avoid the overall test score from becoming negative, a quiz could have another parameter "baseline points." For example, if the test had 100 MCQ's worth 1 point each, a baseline of 25 points would prevent the total score from going below zero. The highest possible score on such a test would be 125 and the lowest possible score would be zero!
I know that on the SAT test, the questions themselves are tested on hundreds of thousands of students before they are actually counted. I'm sure that rigorous item analysis is performed.
I don't really see these as "special grading rules." We already have a penalty factor in adaptive mode. This is also a penalty factor, except that it happens to exceed 1.0 (it would be 1.25 for SAT). I'm not a developer, so I don't understand the coding implications of such a change, but it does seem like a natural extension of the penalty factor. To avoid the overall test score from becoming negative, a quiz could have another parameter "baseline points." For example, if the test had 100 MCQ's worth 1 point each, a baseline of 25 points would prevent the total score from going below zero. The highest possible score on such a test would be 125 and the lowest possible score would be zero!
In reply to John Isner
Re: Need Urgent Help! problem with negatives in quiz module
by Pierre Pichet -
Hi John,
I was very tired when I replied in this forum thread and did not search where I have discussed this negative marking on the forum.
Here it is
http://moodle.org/mod/forum/discuss.php?d=82402
There is also other infos on the docs
http://docs.moodle.org/en/Development:Implementation_of_true_negative_marks_in_MC-type_questions
A more technical discussion on
Rationalising grading options (adaptive, feedback, negative marks, CBM)
with a specific proposal to use the actual quiz structure with a new setting.
Pierre
I was very tired when I replied in this forum thread and did not search where I have discussed this negative marking on the forum.
Here it is
http://moodle.org/mod/forum/discuss.php?d=82402
There is also other infos on the docs
http://docs.moodle.org/en/Development:Implementation_of_true_negative_marks_in_MC-type_questions
A more technical discussion on
Rationalising grading options (adaptive, feedback, negative marks, CBM)
with a specific proposal to use the actual quiz structure with a new setting.
Pierre
In reply to Pierre Pichet
Re: Need Urgent Help! problem with negatives in quiz module
by John Isner -
Hi Pierre,
Thanks for those links. I read them and put them in my links database. I knew the topic had been discussed many times before.
When there are negative marks for MCQ's, the overall quiz score can go negative. But by adding (p-1)N (where p is the penalty factor > 1 and N is the number of MCQ's), the overall score is adjusted to a minimum of zero. AFAIK this is done in all standardized tests that have penalty factors > 1, but I didn't notice that option mentioned in any of the discussions. I don't think anyone wants (or even knows how to interpret) negative overall scores for quizzes.
Thanks for those links. I read them and put them in my links database. I knew the topic had been discussed many times before.
When there are negative marks for MCQ's, the overall quiz score can go negative. But by adding (p-1)N (where p is the penalty factor > 1 and N is the number of MCQ's), the overall score is adjusted to a minimum of zero. AFAIK this is done in all standardized tests that have penalty factors > 1, but I didn't notice that option mentioned in any of the discussions. I don't think anyone wants (or even knows how to interpret) negative overall scores for quizzes.
In reply to John Isner
Re: Need Urgent Help! problem with negatives in quiz module
by Tony Gardner-Medwin -
A sensible strategy for comparing scores that are derived from different question types and different scoring systems is routinely to scale them all so that a perfect set of (confident) correct answers would give 100% and a set of (acknowledged) random guesses would give 0% on average. The adjectives in brackets are to encompass certainty-based (CBM) as well as conventional (correct/incorrect) mark schemes. At UCL and Imperial College these scaled scores get called "% knowledge", and a typical passmark for medical exams tends to be around 50%, whether based on True/False questions (where 50% knowledge corresponds to 75% correct answers) or best-of-5-options (where it corresponds to 60% correct answers) etc..
This scaling is in a way equivalent to the negative marking schemes discussed here, designed to ensure that random guesses give 0% on average. But in one important respect it is I think better not to use explicit negative marking and to do this scaling on the overall score: it avoids the student's dilemma of deciding how much partial knowledge is needed to make it worth answering a question. As people have pointed out, it is in fact nearly always in their best interest to answer a question even with negative marking (unless this is more severe than described). But the psychology of the situation and common bad advice from teachers (not to answer unless fairly sure) may tempt them to omit answers, and on average this will disadvantage them. Advising students not to guess is in my opinion quite unethical in this situation. What you want to do is to reward them for acknowledging when they are uncertain or more or less guessing. Rewarding the correct differentiation between reliable and unreliable answers is the core of CBM.
With all these schemes, overall negative scores are straightforward enough in their interpretation: the student has done worse than they would expect to do by random guessing. A trained monkey would on average do better! Though this ought to be rare in any decent course, there is nothing contradictory or impossible about it. With conventional marking, a badly taught student with misconceptions about the subject may do worse than chance. With CBM, a student who repeatedly claims that incorrect or unreliable answers are definitely right will do worse than someone who correctly acknowledges ignorance or uncertainty. The fact about life is that firm misconceptions are far, far worse than acknowledged ignorance - and deserve to be marked as such! Much of this is what I talked about in the interview about CBM that I posted in a CBM thread yesterday ( www.ucl.ac.uk/lapt/Innovate ).
This scaling is in a way equivalent to the negative marking schemes discussed here, designed to ensure that random guesses give 0% on average. But in one important respect it is I think better not to use explicit negative marking and to do this scaling on the overall score: it avoids the student's dilemma of deciding how much partial knowledge is needed to make it worth answering a question. As people have pointed out, it is in fact nearly always in their best interest to answer a question even with negative marking (unless this is more severe than described). But the psychology of the situation and common bad advice from teachers (not to answer unless fairly sure) may tempt them to omit answers, and on average this will disadvantage them. Advising students not to guess is in my opinion quite unethical in this situation. What you want to do is to reward them for acknowledging when they are uncertain or more or less guessing. Rewarding the correct differentiation between reliable and unreliable answers is the core of CBM.
With all these schemes, overall negative scores are straightforward enough in their interpretation: the student has done worse than they would expect to do by random guessing. A trained monkey would on average do better! Though this ought to be rare in any decent course, there is nothing contradictory or impossible about it. With conventional marking, a badly taught student with misconceptions about the subject may do worse than chance. With CBM, a student who repeatedly claims that incorrect or unreliable answers are definitely right will do worse than someone who correctly acknowledges ignorance or uncertainty. The fact about life is that firm misconceptions are far, far worse than acknowledged ignorance - and deserve to be marked as such! Much of this is what I talked about in the interview about CBM that I posted in a CBM thread yesterday ( www.ucl.ac.uk/lapt/Innovate ).
In reply to Tony Gardner-Medwin
Re: Need Urgent Help! problem with negatives in quiz module
by John Isner -
Thank you for the excellent explanation. I haven't been following the discussions of CBM. Now that I know that it relates to this topic, I will be following those discussions more closely.
Toni, thanks for your advice,
So how do you think Moodle should implement negative or CBM or (perhaps) what should be the questions we need to solve before going to implement negative and CBM?
- From your observations the quiz grading type (regular, negative, CBM) should be clearly identified for the teacher and the student (?).
- When the multichoice question is created or edited is the validation process is different following the grading scheme or all grading parameters are defined at the quiz level?
- Should have new multichoice ( true and false etc.) question types?
- Should we need either new quiz types or a way to select the questions that can be put in the specific quiz?
- Should we show the negative grading value applied?
Or perhaps this is less or more complicated then I think now...
Pierre
In reply to Yatin Kamat
Re: Need Urgent Help! problem with negatives in quiz module
by Vikram Solia -
Yatin,
We implemented negative questions for version 1.6 for the MCQ only. Perhaps it could be done for Moodle 1.8.4 too.
We implemented negative questions for version 1.6 for the MCQ only. Perhaps it could be done for Moodle 1.8.4 too.
In reply to Vikram Solia
Απάντηση: Re: Need Urgent Help! problem with negatives in quiz module
by aggelos panagiotakis -
did you do it ?
can we have your files ?
i posted here a simple modification.... http://moodle.org/mod/forum/discuss.php?d=43552#p407773
can we have your files ?
i posted here a simple modification.... http://moodle.org/mod/forum/discuss.php?d=43552#p407773