Negative Scoring --clarification

Negative Scoring --clarification

by Dave Bethany -
Number of replies: 26

I'm starting a new thread in this, because there seems to be a lot of confusion on what we are all talking about. I found several forum topics that have discussed this, but with no concensus on the problem or the answer.

Negative quiz grading is something that is used quite often in multiple choice, single-answer tests. Perhaps we need a new type of quiz that provides for this.

In the SAT, GRE and CLEP exams lets say that the student gets 1 point for each correct answer. If there are 5 possible answers, then a wrong answer will return a result of -.25 points. This is done to try to eliminate "guessing" or random checking of answers. Thus, in a quiz with 100 multiple choice questions, if the student correctly answers 80 questions they earn 80 points. By answering 20 questions wrong they lose 5 points ( 20 * .25). Thus, their final score is a 75. There is no "penalty" for not answering a question, the student just doesn't get any points for that question. That way, a student could choose to only answer 80 of the 100 questions. If they got 60 correct and 20 wrong, then they earned a 55 (+60 +(20*-.25))

In the existing quiz module, I can assign a negative percentage point value to an individual question, but it has no effect on the outcome. As an example, I create 10 questions, 5 answers each, with each question having a value of 1 point. Correct answer is assigned a +100% value, incorrect is assigned a -25% value. If I take the quiz and answer 6 correct and 4 incorrect, my reported score is 6, where it should be 5.

Thus, anyone who is developing study materials for standardized tests such as SAT or CLEP can not provide an accurate representation of the actual test, simply because they cannot assign a negative value. I have to also believe that there are many users who would want to integrate this type of testing in a normal environment.

Granted, using the adaptive mode I can do penalty points, but this has no relationship to the workings of a true test. They don't actually get the penalty until they retake the question. The student should be able to see a true representative score on a single attempt of a test.

Since the system already displays the negative percentage value in the individual answer areas, I would assume this was considered, but I have found nobody that has been able to make it work.

While I can't see a negative in providing this type of grading in the standard quiz module, perhaps I'm missing something (not unusual for me, ha ha). So, perhaps we can make a secondary module for "standardized testing".

Oh, and I'm using 1.6, but it seems the problem has always existed since there are comments going way back on this in other topic areas.

Thanks all, Dave

Average of ratings: -
In reply to Dave Bethany

Re: Negative Scoring --clarification

by eric eric b. -
This can be fixed by changing a couple lines of code in few php files in your quiz module. 
NOTE: I've only tested this with the multiple choice question type. So if you wish to implement this, testing is recommended.
 In each /mod/quiz/questiontypes/*typehere*/questiontype.php, there is a function called  grade_responses(), in this function they make sure that the grade for each question is not less than zero.  Here is the code:

        // Make sure we don't assign negative or too high marks
        $state->raw_grade = min(max((float) $state->raw_grade,
                            0.0), 1.0) * $question->maxgrade;

You can change it to:
             // Make sure we don't assign negative or too high marks
        $state->raw_grade = min((float) $state->raw_grade,
                            , 1.0) * $question->maxgrade;

This will allow negative points for grades on questions...
You will also need to open up /mod/quiz/locallib.php, to allow for negative points and find the function quiz_apply_penalty_and_timelimit()
There is a line of code:
  
    $state->grade = max($state->grade, $state->last_graded->grade);

If you comment this out then i believe you will get the results you are looking for:

   //$state->grade = max($state->grade, $state->last_graded->grade);

More changes could be made to provide a choice in teh quiz options to allow for this SAT type of question.  Where the above changes would only be implemented when this option is chosen.

Average of ratings: -
In reply to eric eric b.

Re: Negative Scoring --clarification

by eric eric b. -
Actually, there needs to be one more line of code added to mod/quiz/localib.php if you don't want negative total scores. ex. -2.5/10

After or before the line that is commented out in my previous post just add this:
    if($state->grade<0)
    {
        $state->grade=0;
    }


Average of ratings: -
In reply to eric eric b.

Re: Negative Scoring --clarification

by eric eric b. -
Actually that last post doesn't work exactly right, so if the first post is implemented a total of -2.5/10 could be acheived.
Average of ratings: -
In reply to eric eric b.

Re: Negative Scoring --clarification

by Dave Bethany -

Hi Eric and thanks for the assist. Only problem I'm having is that I can't find the "quiz_apply_penalty..."function. Not in locallib.php nor can I find it in any file in the mod directory. Even did a search for "$state->grade ..." can't find it either. I'm using Moodle 1.6.

UPDATE:::

Luckily, I also have an older version of MOODLE running, so I checked the locallib.php file in it and the line does exist. So,

locallib.php,v 1.59.2.41 2006/04/05  the line in question exists (in version 1.5.4)

locallib.php,v 1.95 2006/05/03  the line doesn't exist. (in version 1.6)

Thanks for the help though!!! I knew this could be done and having a negative number is ok for now. If they go negative on the quiz, then they have no business taking the real exam. smile

Dave

Average of ratings: Useful (1)
In reply to Dave Bethany

Re: Negative Scoring --clarification

by Jean-Michel Védrine -
this function is now part of lib/questionlib.php where you vill find the lines:
// Ensure that the grade does not go down
$state->grade = max($state->grade, $state->last_graded->grade);


Average of ratings: -
In reply to Jean-Michel Védrine

Re: Negative Scoring --clarification

by Dave Bethany -

thanks for the additional help, found it with no problem.

Alas, it still doesn't work though. I'll look through the code and see if I can find something else that needs to be changed.

Dave

Average of ratings: -
In reply to Dave Bethany

Re: Negative Scoring --clarification

by Vince Mele -
Many, many thanks for bringing this issue up once again. I too am looking for a way for a penalty for wrong answers in a test that is not adaptive.

Has there been any progress made? Has anyone gotten it to work with Moodle 1.6?
Average of ratings: -
In reply to Dave Bethany

Re: Negative Scoring --clarification

by Vince Mele -
Just to tie this all up, I've got negative scoring working with multiple choice questions. Eric's post above was slightly incorrect by a comma.

I'm using Moodle 1.6+ (2006050506).

1. Edit /question/type/multichoice/questiontype.php
       Change lines 349-350 to: (obviously remove line numbers on the left)
348        // Make sure we don't assign negative or too high marks
349        $state->raw_grade = min((float) $state->raw_grade,
350                             1.0) * $question->maxgrade;

2. Edit /lib/questionlib.php
       Comment out line 1054 so the code looks like this:
1053       // Ensure that the grade does not go down
1054       //   $state->grade = max($state->grade, $state->last_graded->grade);

That's it. Yes, you can have negative scores if someone does bad enough.

Thanks to everyone that has posted on this. The credits go to them.
Average of ratings: -
In reply to Vince Mele

Re: Negative Scoring --clarification

by Nicolas Sicard -
Hi,
Thanks for this hack ! It works well, except I would like to minimize grades to zero. I would like to test for negative grades and set them automatically to zero but I can't find were final grades are computed in the source files sad Any idea ?

nicolas
Average of ratings: -
In reply to Nicolas Sicard

Re: Negative Scoring --clarification

by Anthony Borrow -
Picture of Core developers Picture of Plugin developers Picture of Testers
This sounds like the default behavior of Moodle. I'm confused because it does not sound like negative scoring??
Average of ratings: -
In reply to Anthony Borrow

Re: Negative Scoring --clarification

by Nicolas Sicard -
From what I understand, that's what Moodle does for individual question scoring. As a consequence, the final grade can't be negative (because each question score can't be negative).

But after the code modification, when students have too many wrong answers, their final test grade may become negative, as Vine says :

That's it. Yes, you can have negative scores if someone does bad enough.

What I would like to do is keeping negative scoring for questions (to avoid random answers from students for example) but force global score to zero when negative. I think it should be only a few lines of code to insert, but I can't see where for now sad

Average of ratings: -
In reply to Nicolas Sicard

Re: Negative Scoring --clarification

by Jean-Michel Védrine -
Have you tried to fully implement Eric's ideas and in lib/questionlib.php, after having commented out lines
1053 // Ensure that the grade does not go down
1054 // $state->grade = max($state->grade, $state->last_graded->grade);
to add some code like Eric's original one :
if($state->grade<0)
{
$state->grade=0.0;
}
Or maybe something more condensed like
$state->grade = max($state->grade, 0.0);
I don't really know if it will works but can be worth trying.
Average of ratings: -
In reply to Jean-Michel Védrine

Re: Negative Scoring --clarification

by Anthony Borrow -
Picture of Core developers Picture of Plugin developers Picture of Testers
I think he wants the grade to be able to go down (hence the negative scoring); however, he does not want a total score on the quiz attempt to be less than 0 (if I understand it correctly).
Average of ratings: -
In reply to Nicolas Sicard

Re: Negative Scoring --clarification

by Anthony Borrow -
Picture of Core developers Picture of Plugin developers Picture of Testers
That makes sense. Thanks for clarifying. If it were me (and I have not checked or tried this so proceed at your own risk), I think the simplest thing to do would be to look at the function

quiz_calculate_best_grade($quiz, $attempts) in /mod/quiz/locallib.php

for each of the cases, before returning the value specified, I would do a simple if statement to check if the value is less than 0 and if it is then simply return 0. This allows for the negative score to actually be maintained in the database but should give you the result you want with minimal coding and impact on other aspects. Peace.
Average of ratings: -
In reply to eric eric b.

Re: Negative Scoring --clarification

by Iyappan Krishnaswamy -
Thanks for the code. I has implemented the negative scoring in my moodle project and it works excellent.

Iyappan
Average of ratings: -
In reply to eric eric b.

Re: Negative Scoring --clarification

by Jose C. Molina Castro -

I think there is an error in the code:

$state->raw_grade = min((float) $state->raw_grade,
, 1.0) * $question->maxgrade;

It would be:

$state->raw_grade = min((float) $state->raw_grade,
1.0) * $question->maxgrade;

The difference is a comma.

Average of ratings: -
In reply to Dave Bethany

Re: Negative Scoring --clarification

by Jorge Carvalho Silva -
As I guess that tweaking the quiz code is not an option for the majority of the users, I would suggest that the quiz module should have a new option "Allow negative scoring in multiple choice questions" with default "no" to preserve behaviour of existing quizes but allowing those who want to assign negative scoring to wrong answers to do so. Easier said than done since I don't know how to modify the module's code.
Regards
Jorge
Average of ratings: -
In reply to Jorge Carvalho Silva

Re: Negative Scoring --clarification

by Dave Bethany -

I got sidetracked for the last couple of days, but I'll try to get this figured out this weekend. It really shouldn't be that big of a deal, I just need to figure out how the math is being done and where all of the related code is located.

Dave

Average of ratings: -
In reply to Dave Bethany

Re: Negative Scoring --clarification

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
If you are going to look into this, could you explore the CVS history first. I think I noticed that limiting scores for each question to lie between 0 and 1 was only added quite recently, and it must have been added for a reason.
Average of ratings: -
In reply to Jorge Carvalho Silva

Re: Negative Scoring --clarification

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators
> I would suggest that the quiz module should have a new option "Allow negative scoring in multiple choice questions" with default "no" to preserve behaviour of existing quizes but allowing those who want to assign negative scoring to wrong answers to do so.

I vote for that!

Joseph

Average of ratings: -
In reply to Joseph Rézeau

Re: Negative Scoring --clarification

by Josep M. Fontana -
I don't have time to look for the specific thread and determine whether it was really relevant right now. I seem to remember, though, that there was a discussion with Gustav Delius on this very topic or something closely related (I might have even been involved in it myself  mixed). I was under the impression that this issue had been addressed in 1.6.

Josep M.
Average of ratings: -
In reply to Josep M. Fontana

Re: Negative Scoring --clarification

by Dave Bethany -

Actually 1.6 may be where one of the problems exists. In an earlier post I was given very good directions on how to modify a file, but then learned from another fine contributor that there were at least 2 files involved now. That didn't work, so I'm trying to go through the quiz code to see if I can catch the area needing change.

Something I have said before and will say again, the people that make up this forum are the FINEST that there are. Everyone always works like crazy to try to resolve problems, questions or requests. My requests are usually really weird, but they are always addressed in a fantastic manner. You sure wouldn't get this kind of dedication from a commercial product (you know the one). Heck, WBeBbCBTB (see what a merger will do ) wouldn't even address standard questions, much less off-the-wall ones like mine.

Thanks to all!!

Dave

Average of ratings: -
In reply to Dave Bethany

Re: Negative Scoring --clarification

by Josep M. Fontana -
Sorry, I was wrong. I think I was thinking about the following thread:
http://moodle.org/mod/forum/discuss.php?d=40128#184650

I would have sworn that there was an answer by Gustav Delius saying that he would implement negative scoring outside the adaptive mode. I guess I was wrong. I went back to the thread and it ends without any solutions.

Josep M.
Average of ratings: -
In reply to Josep M. Fontana

Re: Negative Scoring --clarification

by Dave Bethany -

That's one reason why I started a new discussion, this has appeared several times in the past, but always seemed to get sidetracked or lost.

I think there are a number of people that are interested in this and MOODLE has a real opportunity here to make inroads into areas such as standardized testing practice.

With HotPotatoes, you can create negative scores, but only if you go in and manually alter the created HTML files, which is a real pain, because you have to change it on every quiz, for each line. To make it semi-automated you can enter some weird number into one of the quiz development fields so you can do a search and replace, but you still have to be really careful, as HotPot files are large with the potential of using a similar number in an unrelated area.

Average of ratings: -
In reply to Dave Bethany

Re: Negative Scoring --clarification

by acu paul -
long time since no post here, but i'll bump it anyway. what i have seen in other platforms (ilias i think), there was another dropdown for multiple choice, which specified the penalty for not marking that answer.
let's assume you have 4 possible answers and only 2 are correct. so you give each one a 50% of total points, and a negative rating if he does not mark both

in my case, all the answers must be checked for a full grade, otherwise it should be marked with 0 points. not marking the right choice will still give them 33 or 50% of the total score, and that's inapropriate

ayone needed this in their quizez?
Average of ratings: -
In reply to acu paul

Re: Negative Scoring --clarification

by Thomas Eibel -

I overcome this in most cases by putting both correct answers in one field and vary it with others. E.g.:
What is the neccessary to get good grades?
a) good homework and good test
b) good test and blue eyes
c) blue eyes are sufficient
d) good homework and a friendly behaviour
e) good tests are sufficient

best regards, thomas

Average of ratings: -