There are a few modes / combinations that our institution would like to see supported:
- sequential n-marking: Assignment must be marked by at least 'n' markers. In this case the number of markers may be known, but who they are marking and in which order isn't.
- sequential n-marking with the same markers for all submissions: Similar to (1) but the markers to be allocated and their order is already known.
- parallel versions of 1 & 2. In this case the markers can do it at the same time.
- Blind & non-blind marking between markers: Markers may or may not see what other markers have submitted.
- Moderation: An 'n+1' marker who can see all of the previous markings and may:
- Select a grading to use as the final grade.
- Amend the final grade
- Provide "global" feedback
With regard to Tim's observations:
2) This is 2 qustions. 1) Default No: The visibility between markers should be up to the teacher setting up the activity. For us in most cases this would be no markers can't collaborate on the marking it should be reached independently. Special case being the "last" / moderating marker.
3) Part of me is going with a manual reconcilation process, and this may be only way to do it with some of the adv. grading methods. If assignment is using simple directed grading, then the options similar to the grade book could be offered: Average, Highest Mark, Lowest etc.
4) Again this would be up to the staff. In many cases such as straight up assignments the final synthesised grade and feedback is the only thing to be presented to the students, but in others such as dissertations, having the separated individual markers results may be useful.
The final thing that comes up in my mind is the impact with Adv. Grading methods.
If these are configured then you'd expect each of the markers to be presented with the appropriate form.
This complicates the manual reconcilation process, especially if you think of something like the rubric.
The "grade" part is fairly simple, allow the moderator / last grader to
specify the value (i.e. it's up to them to work it out), however the "feedback" is more problematic.
One option could be to present all of the forms side by side, with a "final" form at the bottom for the moderating user to complete based on the previous markings. Or as a "quick" mechanism allow the moderating user to select one of the previous markings as the basis for the final marking.
That's my brain dump of my thoughts from over the last 4 years of our staff asking...