I copied my current 1.9.9 server to test 2.0. I upgraded it and when I went to check out the new layouts for quiz reports, I noticed the "Dry Run" option for regrades. I tested it out on a past test and suddenly it said that 7 out of 20 students needed to be regraded. When I looked at the responses it wanted to regrade, it was trying to give the student full marks when they had gotten half marks (1/2 mark penalty). Also if they had gotten it incorrect, but gotten the correct answer after 4 or 5 attempts (at that point all possible credit was gone due to 1/2 mark penalty and the score was 0) the regrade wanted to give them full marks.
It didn't do this for all questions, only a few.
I made sure apply penalties was still turned on for the test settings, it was. I also made sure the questions in question had a penalty value, they did.
Regrades were done after the test was originally given in 1.9.9. Why would 2.0 suddenly try to reverse that and give credit back? The questions and tests weren't edited in anyway after the moodle upgrade.
In reply to K P
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by Tim Hunt -
Well, that sounds like a bug.
Can you come up with a minimal test-case. That is, instructions for creating a quiz with particular settings with just one question with particular settings, and then logging in as a student and entering certain data, and then logging in as teacher and regrading.
I did not think the regrade code had changed significantly in Moodle 2.0, but something must have gone wrong. Anyway, it will be must easier to fix if you can tell me how to reproduce it "under lab conditions" as it were.
Can you come up with a minimal test-case. That is, instructions for creating a quiz with particular settings with just one question with particular settings, and then logging in as a student and entering certain data, and then logging in as teacher and regrading.
I did not think the regrade code had changed significantly in Moodle 2.0, but something must have gone wrong. Anyway, it will be must easier to fix if you can tell me how to reproduce it "under lab conditions" as it were.
In reply to Tim Hunt
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by K P -
I figured it out and it would affect anyone upgrading to 2.0 from an earlier version who have calculated questions where they also graded for a specific unit. It seems that that the regrade issue is limited question types that have had new features added that affect how the question is graded. In moodle 1.9.9 the units are not graded as a separate item so if a student gets the wrong unit but the correct numeric answer, the penalty kicks in for the whole response. Because 2.0 grades the unit separately, you can have a separate penalty for the unit or tell it not to grade units at all. In an upgrade when the questions are converted to the new calculated question format with unit handling, unit grading is defaulted OFF. So 2.0 is regrading the questions using the default setting which didn't exist in earlier versions.
Questions created in 2.0 aren't affected, only upgraded questions.
I am not sure how many other question types have changed in 2.0 and now include new features that didn't exist in pre-2.0 versions, but I would assume that a regrade in 2.0 after an upgraded Moodle might result in the same type of situation.
So, really this is only a problem if you happened to do a regrade of a 1.9 quiz after an update to 2.0. In that case, I guess you would have to go back and manually turn on unit grading on each calculated question so the marks don't change.
In reply to K P
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by Paul Nicholls -
I'd consider this to be a bug - the upgrade process should surely default this new setting for upgraded questions to whatever state allows the behaviour to remain consistent with how it was pre-upgrade (upgrading Moodle shouldn't cause existing quizzes to be graded differently!).
If you haven't already done so, I'd suggest creating a ticket in the tracker for this (if there isn't already one - please search first) - if you paste a link to here, I'm sure people who have been following this thread will vote on your ticket (I sure will). Hopefully, a ticket will get attention from those who know what has changed for various question types, and we'll end up with Moodle 2.0 regrading upgraded quizzes exactly the same as Moodle 1.9 - no matter what.
Edit: I'm not necessarily advocating changing the defaults for these settings when creating new questions - just the setting used by the upgrade process. What the defaults should be for new questions is another matter entirely...
If you haven't already done so, I'd suggest creating a ticket in the tracker for this (if there isn't already one - please search first) - if you paste a link to here, I'm sure people who have been following this thread will vote on your ticket (I sure will). Hopefully, a ticket will get attention from those who know what has changed for various question types, and we'll end up with Moodle 2.0 regrading upgraded quizzes exactly the same as Moodle 1.9 - no matter what.
Edit: I'm not necessarily advocating changing the defaults for these settings when creating new questions - just the setting used by the upgrade process. What the defaults should be for new questions is another matter entirely...
In reply to Paul Nicholls
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by K P -
Well I the problem is that the defaulted item didn't even exist in earlier versions.
I can see it as a bug, but then again I can see it not as a bug.
2.0 can't really "maintain" the old question settings if that setting didn't exist. The fact that it defaulted to off was a bad thing for me. If the 2.0 default was changed to turn unit grading on, then people who didn't penalize students for wrong units before might find themselves in the same situation if they did a regrade on the quiz after an upgrade.
So it seems a catch 22 with the default setting. Even though it was a surprise and it took me a bit to figure out, I am not sure it warrants a tracker listing as a bug. Before I knew what the issue was, I did figure it was a bug, but how can you code to set a default value for a question to match something that doesn't exist in the database to compare it to? get me?
I think that this should be more of a warning prior to upgrading...but there is already so much information being included on the upgrade page that I could see it being completely overlooked. I know I wouldn't have saw a warning that said...."oh hey if you upgrade, don't do a regrade because default settings might cause a problem for your old graded questions"
I can see it as a bug, but then again I can see it not as a bug.
2.0 can't really "maintain" the old question settings if that setting didn't exist. The fact that it defaulted to off was a bad thing for me. If the 2.0 default was changed to turn unit grading on, then people who didn't penalize students for wrong units before might find themselves in the same situation if they did a regrade on the quiz after an upgrade.
So it seems a catch 22 with the default setting. Even though it was a surprise and it took me a bit to figure out, I am not sure it warrants a tracker listing as a bug. Before I knew what the issue was, I did figure it was a bug, but how can you code to set a default value for a question to match something that doesn't exist in the database to compare it to? get me?
I think that this should be more of a warning prior to upgrading...but there is already so much information being included on the upgrade page that I could see it being completely overlooked. I know I wouldn't have saw a warning that said...."oh hey if you upgrade, don't do a regrade because default settings might cause a problem for your old graded questions"
In reply to K P
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by Paul Nicholls -
Hm - sounds like this whole thing really needs some more thought. Perhaps a better approach would be to have a warning which appears before you can regrade an upgraded quiz, which lists any relevant changes and provides an opportunity to cancel the regrade and make any adjustments required to maintain the original behaviour before proceeding with a regrade. If it could be rigged to only show up on quizzes with affected question types, that'd be even better...
In reply to Paul Nicholls
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by K P -
Well the "Dry Run" does just that. It tells you what changes will be made and then asks you if in fact you want to continue or not. That is how I caught this originally, because it shows you the old grade, the new grade and even tells you that a particular attempt needs regrading.
It just didn't jump right out at me why the regrade was going to give full marks. The issue only occurs when you have upgraded your server to 2.0 and you have older quizzes. Maybe a possible solution could be if it detects that the quiz or questions were made with an older version of Moodle it would give a warning about how the upgrade includes new defaults that didn't exist before and a regrade may undesirably change the marks. But that would only work if the quiz/questions have a version tag associated with them.
I just think that when 2.0 comes out as a production solution, there might be quite a few angry teachers. For me, I wouldn't really do a regrade of a past test unless a student had missed the test and had to make it up. If I upgraded to 2.0 between when the test was given and when absent student make up a test/quiz and I wanted to regrade their test, and then discovered that all the other student's marks were going to change, I would be pretty upset and confused at what was happening (I think Moodle 2.0 allows to select specific attempts to regrade so made this is the solution along with a warning). I don't give students access to their quiz scores for a few reasons, so I have no fear of a student seeing the new "regraded" score and question it. If teachers use the gradebook for keeping real course grades and/or students can see their scores this issue would be a major thing. I know my students/parents would call me out on why test scores were showing up with a higher value then what I put in the official gradebook program (pre moodle upgrade score)
It just didn't jump right out at me why the regrade was going to give full marks. The issue only occurs when you have upgraded your server to 2.0 and you have older quizzes. Maybe a possible solution could be if it detects that the quiz or questions were made with an older version of Moodle it would give a warning about how the upgrade includes new defaults that didn't exist before and a regrade may undesirably change the marks. But that would only work if the quiz/questions have a version tag associated with them.
I just think that when 2.0 comes out as a production solution, there might be quite a few angry teachers. For me, I wouldn't really do a regrade of a past test unless a student had missed the test and had to make it up. If I upgraded to 2.0 between when the test was given and when absent student make up a test/quiz and I wanted to regrade their test, and then discovered that all the other student's marks were going to change, I would be pretty upset and confused at what was happening (I think Moodle 2.0 allows to select specific attempts to regrade so made this is the solution along with a warning). I don't give students access to their quiz scores for a few reasons, so I have no fear of a student seeing the new "regraded" score and question it. If teachers use the gradebook for keeping real course grades and/or students can see their scores this issue would be a major thing. I know my students/parents would call me out on why test scores were showing up with a higher value then what I put in the official gradebook program (pre moodle upgrade score)
In reply to K P
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by Tim Hunt -
Ideally:
1. Student takes quiz in 1.9
2. Admin upgrades to 2.0
3. Teacher re-grades quiz
should not cause the score to change. Please could you create an issue at http://tracker.moodle.org/ and I will investigate.
In reply to Tim Hunt
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by Pierre Pichet -
Previous to 2,0, if the teacher does not define any unit and if the student put any units in their response, their grade was 0.
And this feature is not necessarily known by the teacher that do not define a unit.If their students do not put any units in their response, they will have all their full grade.
On the other end, if a teacher define a unit and the student just put the numerical, he receive a full grade in 1.9 and previous versions.
We could consider that 1,9 and previous were bugged...
I have to adjust 2,0 code to this feature which seems almost a bug in 2,0 as the number and unit field will be clearly defined and the unit field shown only if the unit is graded.
I have to look
- how I can transfer the 1.9 thruth table in 2,0
- how I can detect old questions and apply the correct settings in migrating from 1,9 to 2,0 either by the backup or by import.
Actually these backup and import processes are not solved for 2,0.
Pierre
P.S. Tim, could you migrate this discussion on the question forum.
In reply to Tim Hunt
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by Pierre Pichet -
The case of 0 grade if no units are defined by the teacher, and any units (any other character than space) is added by the student can be equivalent to 100% penalty on total grade when no units defined. (the actual 2,0 code needs correction)
This could be set as default but should be corrected when reediting the question by setting the validation code correctly.
Actually in 2,0, the restore differentiate the old and new attempts as they are stored differently and I had checked the code so that the grades were not changed although I forgot this option. This is a bug in 2,0 that I can fix.
The case of giving full grade if there is a unit defined (ex:5.6 cm3) but the student does not write any unit (ex:5.3) could be maintain in 2,0 but appears incompatible with the 0% grade if the student badly wrote the unit (ex:5.6 cm2).
Students quickly learn to avoid adding units...
This is where 1,9 grading is somehow an historical bug.
The best way could be to add a warning on the existing quiz previewing to invite teacher to use the full features of the new interface.
Pierre
In reply to Pierre Pichet
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by K P -
"Students quickly learn to avoid adding units..."
This was exactly my problem in 1.9. If I defined a unit of "m/s", but a student put "m" they got the penalty, but if a student added no unit to their numerical answer, they got full credit. This was not fair because I wanted the unit to be graded and I considered no unit to need a penalty, but 1.9 wouldn't penalize. In the end, I had to create two questions: 1) A calculated question asking for only a numeric answer, and 2) A short answer question asking for the unit associated with the above numerical answer. The new issue this created was that I can no longer shuffle questions.
In reply to K P
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by Pierre Pichet -
This is one of the reasons for the new 2,0 handling of numerical answers by grading the number and then applying a penalty if the unit is not OK.
Teacher won't have to create 2 questions as you did.
However your two 1,9 questions cannot be fused automatically in just one 2,0 question
I just forgot to look for users that could put a unit in the number input element even when clearly identify as a number in 2,0
Pierre
P.S. still working on MDL-24504
In reply to Tim Hunt
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by Pierre Pichet -
Effectively in the new interface I forget about giving 0 grade for a student that put a unit in the number field when there are no units defined as I never used this feature i.e. I always define a unit when there was one and do not worry if the student don't put one in his answer. He will not be penalize.
Furthermore units in chemistry often implied exponents that could not be render correctly.
In Moodle the units handling was devised to allow the students to put his numerical value in the units of his choice as long as the teacher defined them.
If the teacher define units, he has to put in the question text the available units.
So I have to revise the 2,0 code to handle the case that the student put an unit in the number element and default it to full error (penalty = 1,0) when he will attempt the quiz.
This can be done and imply changes in the question editing interface as well as setting the default values.
This should be done for next week.
The case of giving full grade if the student don't write any units if units are defined is different.
On old attempts this grading anomaly should remain.
On new attempts the new interface with two input element should appears and the student will need to put the correct unit. The default penalty should be 1,0 i.e. 0 grade if no units are written or badly written.
This means that teacher should revised their questions if they are not satisfied with the default new 2,0 unit handling.
The 1,9 anomaly has no equivalent in 2,0 as 2,0 offer new ways to handle units that are more conform to standard grading practice.
OR
We maintain two ways to handle number and units.
I don't think so.
Pierre
In reply to Tim Hunt
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by Pierre Pichet -
The 1,9 - 2,0 dilemna of applying full penalty when there is no unit defined and the student put a unit ( any alpha after the number) could be solved if we modify the 2,0 so that the unit penalty is always applied.
- when units are defined as in 1,9, the number element and the unit element appears side-by-side so there should be no confusion and the student must not put any units in the number element. This is valid for 2,0 and also for migrated 1,9 questions which will appears as regular 2,0 with the 2 elements.
- when there is no unit defined, only the number element is shown and the user either 1,9 or 2,0 should not put any units (any alpha after the number). If he does, the penalty will be applied. This penalty is default to 1,0 with old questions.
So I need to
- modify the unit edit interface so that clearly the unit penalty is always applied
- old questions have 1,0 penalty default either when backup or when import.
- setting correctly the internal grading code.
This should render the migration transparent. ;)
Pierre
In reply to K P
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by Joseph Rézeau -
Hi Kameron,
I certainly would not envisage upgrading from 1.9 to 2.0 in the middle of an academic year, with some quizzes still to be taken by students, regrading problems etc. Upgrading to a major version should really be done during vacation time, and all courses be reset upon using 2.0 at the beginning of the new academic year.
However, I agree that this ideal scenario may not be possible in some teaching contexts.
Joseph
In reply to Joseph Rézeau
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by K P -
I haven't upgraded my main site yet. I only upgraded a copy of the site to test. There are still major issues with other things such as no horizontal scroll bars while looking at the quiz reports.
I just happened on this issue.
In reply to Joseph Rézeau
Re: Regrade is reversing penalty for quiz questions for upgraded 1.9.9 to 2.0
by Jessica Gramp -
Unfortunately some of us have to contend with courses that run outside of standard academic term times, so these quiz issues are causing us some real headaches - also not all courses are reset at the end of the teaching period and start fresh each year - some are ongoing...
Hopefully these regrading issues can be sorted out or at least workarounds discovered...