I've just released a plugin to crowd source and automate the grading of submissions to the Moodle assignment (mod_assign) activity working with Ian Jones at Loughborough University.
By enabling this submission plugin (alongside online text, files etc.) users will, once certain criteria have been met, be prompted to compare pairs of submissions, make comments and choose the better of the two. Once enough of these comparisons have been made they can be fed through an algorithm to automatically grade the assignments, these grades and comments can then be automatically pushed into the grade book.
Judges can be chosen by selecting one or more roles within the context of the course or users who have submitted an assignment.
Emails can be sent to remind those who need to compare some submissions to visit the site and do so.
I've submitted everything to the plugins db for review and inclusion but in the mean-time you can get to everything here:
https://www.opensourcelearning.co.uk/assignsubmission-comparativejudgeme