I've read this before, and experimented with the grading outcome. If the process is deterministic, then it can be specified with parameters and, if not a formula, then code. Is it possible to see the code on this? The determination of the top score is clear enough. What is missing is how the distance to ideal and therefore the degredation of grades is mapped to a declining score. I remember there being a "strictness" scale as far as how well one converges to the ideal score. Can there be some more explanation of how that function works? Is this linear? Or does it work more like OLS, where big deviations are "penalized" more?
Re: grading algorithm for workshop - control on reveal feedback by Eric Lin -Wednesday, May 20, 2020, 11:14 PM
by Eric Lin -
Number of replies: 1
In reply to Eric Lin
Re: grading algorithm for workshop - control on reveal feedback by Eric Lin -Wednesday, May 20, 2020, 11:14 PM
by David Mudrák -
Is it possible to see the code on this?
Of course it is - see the file mod/workshop/eval/best/lib.php around the line no. 150 - the method process_assessments() and other methods it uses such as average_assessment(), weighted_variance() or assessments_distance().
There are also unit tests for these, and it should not be hard to even produce some kind of charts and tables illustrating the behavior of the algorithm.
One of my long-term nice-to-have features for workshop is adding support for radar / spider charts that would display how one submission was assessed by multiple reviewers (one reviewer = one colored line around the web), each one assessing multiple criteria (each direction in the chart representing one criterion). It would then allow to show the hypothetical "best" (average) assessment and how the student's one is different from it.