Grade for assessment calculation

Grade for assessment calculation

by Frank Koch -
Number of replies: 4
Picture of Plugin developers

My observation refers to Moodle Doc Using Workshop. There it states:

Grade for assessment
...
Grades for assessment are displayed in the braces () in the Workshop grades report. The final grade for assessment is calculated as the average of particular grading grades.
There is not a single formula to describe the calculation. However the process is deterministic. Workshop picks one of the assessments as the best one - that is closest to the mean of all assessments - and gives it 100% grade. Then it measures a 'distance' of all other assessments from this best one and gives them the lower grade, the more different they are from the best (given that the best one represents a consensus of the majority of assessors). The parameter of the calculation is how strict we should be, that is how quickly the grades fall down if they differ from the best one. 

I tried to understand the "grades for assessment" with the following data (based on 80% uncategorised for grade for submission, 20% uncategorised for grade for assessment, and "very strict" comparison of assessments:

test data

  • The 1st row is OK to receive 20, cause 53.33 is closest to the mean.
  • The 2nd row is OK to receive 20, cause 48.00 is closest to the mean.
  • The 3rd row is OK to receive 20, cause 26.66 is closest to the mean.
  • The 4th row shows the problem: why the heck does the circled 58.66 receive the 20, even though 56 is closer to the mean?

Could someone please reveal the formula behind this calculation? Or is it maybe even a bug?

Thank you very much for your help.

Frank







Average of ratings: -
In reply to Frank Koch

Re: Grade for assessment calculation

by David Mudrák -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators
It's important to read whole that docs page. Please see the "It's not final grades what is compared" section at https://docs.moodle.org/27/en/Using_Workshop#Grade_for_assessment I believe it will clarify things.

Also note, the comparison with the best assessment is not the only evaluation method available. See https://moodle.org/plugins/browse.php?list=category&id=33 for alternatives.
Average of ratings: Useful (1)
In reply to David Mudrák

Re: Grade for assessment calculation

by Frank Koch -
Picture of Plugin developers

Thank you David. That helped and now I understand why the circled 58.66 received the 20 smile

However, even though I digged deeper and have better understanding, I still cannot explain the calculation behind the grade for assessment (e.g. 9.21010 or 8.70715 in the given example). Then I discovered in https://moodle.org/mod/forum/discuss.php?d=109988 your comment "The way how Workshop calculates the grades is dark magic..." and furthermore "I am currently working on the Workshop rewrite and these things make me mad...". Well, that was 2009! Are you still working on this issue and can we expect some more transparency coming up? Or would it be possible to see the documentation of that magic code?

I'm introducing peer assessment to faculty and of course they need to be very capable to explain the given grades to students.

Thanks again for our help

Frank

Average of ratings: -
In reply to Frank Koch

Re: Grade for assessment calculation

by David Mudrák -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators
That is a nice reminder smile That post is from times when I was rewriting the Workshop 1.x into the current 2.x version and I (quite naively) thought there would be much simpler way how to calculate the grade for assessment. It turned out to be not trivial at all. So the result of that work was 1) making the calculation pluggable so that alternative methods can be simply used and 2) making the default method (which is what you use) as backwards compatible as possible, yet improved in some special cases that the previous version did not handle well or predictably.

The only available docs beside the one at https://docs.moodle.org/en/Using_Workshop#Grade_for_assessment is the original behaviour specification at https://docs.moodle.org/dev/Workshop_2.0_specification#Grade_for_assessment that describes the process of the grade calculation. The actual implementation may be slightly different in details but the overall concept is the same.

What the workshop is still missing (among other things) is the reporting feature mentioned in the spec. That should provide a clearer explanation of how the grade was calculated. There are no particular plans on that one for the near future though.

As I mentioned above, you may be interested in using the alternative grade calculation plugins available in the Moodle plugins directory.

HTH
Average of ratings: -
In reply to David Mudrák

Re: Grade for assessment calculation

by Annette Knobloch -

I went to this link. The download looks like a program, and not a narrative description. A download is probably useless to me, individually.

Average of ratings: -