Hi Mark
First of all, thanks for the testing. I will try to explain using your screenshot as the example.
I can see there are three students participating in this workshop. Each of them was asked to assess the work submitted by the other twos. In Workshop 2.0, every participant gets two grades into the course gradebook. The first one is the grade for the submission that measures the quality of the submitted work. In your workshop, this grade for submission can be out of 80. The second grade is a grade for assessment that tries to estimate the quality of assessments that the participant gave to the peers. The calculation of this grade for assessments (also known as "grading grade") is calculated by the artificial intelligence hidden within the workshop as it tries to do teacher's job. In your workshop, the grade for assessment is out of 20.
Now, let us look at the figures displayed. Student1 submitted her work called "fgbgfhy". This submission was assessed by her peers - Student2 and Student3. Student2 gave 51/80 and Student3 gave 80/80. The total grade for Student1's submission is the average, that is 51+80/2 = 65 (rounded - you can increase the number of decimals displayed to get the more precise values). The values from which the total grades are calculated are in bold for every participant.
Student1 was asked to assess two submissions. She gave 40/80 to Student2's submission and 59/80 to Student3's submission - see the column "Grades given". The symbols < and > should help you to realize if the grade comes from a peer or is to given to a peer.
Now, during the Grading evaluation phase, workshop tried to calculate grades for assessment. These grades are put in the braces () in the table. In your workshop, every student got full 20/20 for all their assessments and there their average is 20 again. So the total grade for assessment for Student1 (in the last column) is calculated as the average of two grades 20. Look at one before the last column - the Grades given - for Student1. There are 40(20) and 59(20). You should read this as: Student1 gave grade 40 and the grade for this assessment was calculated to 20. The same for 59(20). Only the bold values - 20 - are used for the average calculation in the last column.
The last thing to explain is why workshop decided that all participants should have full grades for assessment, that is 20/20? There is not a single formula but the process is deterministic and I am going to describe it in Workshop documentation ASAP. Shortly, Workshop picks one of the assessments as the best one - that is closest to the mean of all assessments - and gives it 100% grade. Then it measures a "distance" of all other assessments from this best one and gives them the lower grade, the more different they are from the best (given that the best one represents a consensus of the majority of assessors). The parameter of the calculation is how strict we should be, that is how quickly the grades fall down if they are different from the best one.
But in your case, you have just two assessments per submission. So Workshop can not decide which of them is "correct". Imagine you have two reviewers - Alice and Bob. They both assess Cindy's submission. Alice says it is rubbish and Bob says it is excellent. There is no way how to decide this. So workshop simply says - ok, you both are right and I will give you both 100% grade for you assessment. That is why all your students have 20/20. To fix it, you have two options:
- Either you must provide an additional assessment so the number of assessors (reviewers) is odd and workshop will be able to pick the best one. Typically, you as the teacher come and provide your own assessment of the submission to judge it
- Or you may decide that you trust one of the reviewers more. For example you know that Alice is much better in assessing than Bob is. In that case, you can increase the weight of Alice's assessment, let us say to "2" (instead of default "1"). For the purposes of calculation, Alice's assessment will be considered as if there were two reviewers having the exactly same opinion and therefore it is likely to be picked as the best one.
You can combine these two options, obviously.
I hope this helped to clarify a bit. See http://docs.moodle.org/en/Workshop_module for more information, I am going to update it soon.
p.s. by the way, in Workshop 1.x this case of exactly two assessors with the same weight is not handled properly and leads to wrong results as only the one of them is lucky to get 100% and the second get lower grade.
p.p.s. note that the calculation of grades for assessment is now pluggable in workshop so there may be more advanced/different methods implemented in the future, even the custom ones
p.p.p.s. it is good to hightlight that there is nothing special about teacher assessments. If you want or need, just set a higher weight for your own assessment (as the teacher's one).
