Frustrations: Moodle & Blinded Grading

Frustrations: Moodle & Blinded Grading

by Eric Kennedy -
Number of replies: 9

Just wanted to share a front-line frustration from trying to mangle Moodle (3.5.6) into actually allowing blinded grading.

When operating in blinded grading, one cannot reveal student grades without unblinding the assignments. This is frustrating, because when I'm running multiple stage assignments (e.g., they submit short essays in serial; receiving feedback on each), it forces me to reveal identities to release grades. This is frustrating, because I don't want to see who is writing on what topic, or who received what grade - I want the entire assignment process to remain blinded until the end of the project. 

Per https://docs.moodle.org/37/en/Assignment_settings#Anonymous_marking, this is normal system behaviour, so I suppose this is a feature request to allow those of us who actually want to maintain blinding throughout an assignment or semester to do so.

Average of ratings: Useful (1)
In reply to Eric Kennedy

Re: Frustrations: Moodle & Blinded Grading

by Eric Kennedy -
Oh, and there appears to be no way to grant extensions to students under the blinded system either...
In reply to Eric Kennedy

Re: Frustrations: Moodle & Blinded Grading

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
I don't use blind grading, but your post got me wondering. If you were to grant an extension to a student, and then went to grade the quiz after the due date, wouldn't you then know who that student is? Wouldn't it defeat the "blind" grading desire?  So maybe Moodle's logic is correct for blind grading?

(Sorry if I am way off base from what you are trying to do, or say.)
In reply to Rick Jerz

Re: Frustrations: Moodle & Blinded Grading

by Eric Kennedy -
No, it's a totally reasonable question!

The workflow I use is that I choose date X as the day I plan to actually grade assignments, and then make them due on date X-1. But, everyone in my class has a free 24hr extension, as long as they notify me before the due date. So, what I'd like to be able to do is to go in + record who took the extension in Moodle as they request them, then be able to see if it was on time or not (e.g., day X-1 if no extension, day X if extension) while still in blinded mode.

As an aside, the workflow for the first request is that we have a three-part assignment where they select a common case study throughout the whole term. I'd like to be able to grade all three parts blinded before finally revealing identities at the end of the semester (i.e., go back and see what I said about assignment 1 when grading assignment 2). Of course, some students discuss their topics with me... that's fine, and I mostly forget who is who anyways. But, the more I can keep it blinded, the better.
In reply to Eric Kennedy

Re: Frustrations: Moodle & Blinded Grading

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
So we might be back to my original question. If only one (or two) students ask for the extension, they when you go to grade it after the due date, you know who this student is. This is no longer blind. (Even if only two or three students ask for the extension, it really is not quite blind.)

Or, if you find that most students ask for the extension, why not simply set the due date to your cut-off. Why make nearly all students ask, knowing that they are all granted a one-day extension?

Also, since you allow students to discuss what they are doing, you probably are not totally unbiased. If you forget who is who, why worry about blind grading? Do you feel that you really can't be objective when you see their name? Do you really mean that the most beautiful young attractive student whose name you have memorized is going to affect your grading of her assignment... that you can't force yourself to be objective? If what I ask is true, why not consider using a rubric so that you can blast through the grading of these assignments with fixed words (from your rubric?)

Just as a point of reference, I have around 125 college students spread between three online classes. When students submit an assignment, I can't figure out who is who. This is probably why I don't use blind grading. I always grade offline, too.

If you don't mind me asking, what kind of assignments are they submitting? Word files? Excel? Artwork? And what kind of topic, such as "Explain the most embarrassing thing that happened to you when you were 14?" Or "create a graph of airplane delays over the last year?" Is this a "law" course? I have heard law schools being concerned about blind grading.

Well, as you can see, I don't know exactly what you are doing, but maybe my questions will still help you think through your situation.
In reply to Rick Jerz

Re: Frustrations: Moodle & Blinded Grading

by Eric Kennedy -
Hi Rick,

The typical response is that somewhere between 10-20 in a class of 60 ask for the 24-hr extension. So, it's a nice level where I functionally don't know who asks for it, but it's also not the entire class. Hence my question of wanting to be able to note these during blind mode, because otherwise I have to grade, then unblind, then adjust grades based on who was actually late vs. not.

Re: "why worry about blind grading," I do it because I am a human. Like all humans, I'm certainly not unbiased. I use blind grading because there is strong evidence that professors are subject to the exact same biases everyone else is. We know, for instance, that professors use significantly fewer superlatives in describing female students than male ones (Schmader et al, 2007). Meta reviews show that bias affects all level of grading, including university and collegiate, and that professors and instructors take into account much information other than the quality of the work in their grades assigned (Malouff & Thorsteinsson, 2016). And, we also know - to my specific feature request about keeping assignments blinded throughout the term - that performance on one assignment statistically affects our grading of the student's other subsequent assignments, which is known as the "halo effect" (Malouff et al, 2013).

The reason to use blind grading consistently is that our grading biases can creep in via many different pathways. In your classes summing to 125 or my class of 60, it's not that bias only occurs if you know everyone's assignment... even if you just remember that one keen, engaged, high-performing student's topic, you're likely to subconsciously grade them higher because of the halo effect.

I do use rubrics, by the way. However, the problem with bias is that it affects /how/ we use the rubric... that we don't use it consistently from student to student.

Of course, as I mentioned earlier, there's a challenge: despite the strong body of literature demonstrating the vital importance of blind grading in all assignments by all instructors, personalized mentorship is just as important (e.g., feedback on topics, etc). This is where trying to explicitly fight to control biases comes in (acknowledging each of our limitations as a human, rather than claiming false objectivity), as well as trying to get participation up to such a point where I can overload my memory and try to avoid remembering their topics.

Because you asked, they're submitting essays in this course on cognitive science. But I also do it in my ethics class and research methods class too - all three being places where "walking the walk" of being fair and empirically informed in my pedagogical methods is important.

Hence my feature request.

Best,

Eric


Works Cited:
Malouff, J. M., Emmerton, A. J., & Schutte, N. S. (2013). The risk of a halo bias as a reason to keep students anonymous during grading. Teaching of Psychology, 40(3), 233-237.

Malouff, J. M., & Thorsteinsson, E. B. (2016). Bias in grading: A meta-analysis of experimental research findings. Australian Journal of Education, 60(3), 245-256.

Schmader, T., Whitehead, J., & Wysocki, V. H. (2007). A linguistic comparison of letters of recommendation for male and female chemistry and biochemistry job applicants. Sex roles, 57(7-8), 509-514.
In reply to Eric Kennedy

Re: Frustrations: Moodle & Blinded Grading

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
Yes, I understand. However, we are still human beings. I understand the research. But there is still a practical issue.

In the U.S., when our president Trump tweets a simple paragraph, some people rate it high, and some low. Is this because people know the source?
However, even if you didn't know the source, some would still rate it high and some low. This is because we are still humans, and we are each individually interpret what was said. Sure, you can add all kinds of overhead to this grading, like having a team of 20 instructors collectively grade each student's essay question. But this is a lot of overhead. Quite honestly, this is why I limit my use of assignments. They are really hard to grade. The best that I am able to do is to look for key items in an assignment.

I teach business courses. The problem in business is that success can happen to non-standard methods. My guess is that Bill Gates and the (late) Steve Jobs would get bad grades for their business ideas because they were not replying with the "standard" or "expected" response. Of course, I love teaching quantitative business courses because 2+3 is almost always equal to ... 6? No, 5.

And then, in my case, I also have international students. I can often tell that a forum post is from an international student, even before I look at their names. How do you remove this kind of bias?

Anyway, I appreciate what you are trying to do. I am enjoying this discussion. As you can see, I am far from the expert on blind grading.

Getting back to Moodle... Why not create two assignments, one titled "1st submission" and then another "2nd resubmission?" Then, in the grade book, create a grade item that would take the "maximum" of these two assignments? It wouldn't take long to do this.  (Incidentally, I think Moodle is the only LMS that can do this.)

(Sorry that our discussion has not attracted others to comment.)
In reply to Rick Jerz

Re: Frustrations: Moodle & Blinded Grading

by Eric Kennedy -

Just jumping back here to add a solution in case any other forum users come along in hopes of being able to do this. It works!

If you click on the assignment to get to the summary page (i.e., the one with the due date, number submitted, etc) and click the gear icon in the top right, there's an option called "User Overrides". This allows you to adjust individual due dates by name, while /not/ unblinding the grading. That way, it will test whether each user was "on time" according to their individual deadline, but won't reveal who is who.

As Rick mentioned above, if you have a class of 8 and only one requests an extension, of course, this won't be de facto blinded. But in larger classes where multiple people request extensions, it lets you grant them within the Moodle interface without having to unblind assignments.

In reply to Eric Kennedy

Re: Frustrations: Moodle & Blinded Grading

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
Great! Someday I might have to explore this. I forgot that Assignments allows user overrides.