After a week of searching, we finally found the code that this takes place. What we originally considered the bug was a single if statement. We discovered that if we commented it out, then the proper amount of assessments would appear for all of the students. However, we soon realized that removing the if statement all together could result in a new and perhaps larger problem.
It appeared that the new problem could result in a student who would end up not only having no assessments to assess, but also that nobody would assess their work.
Consider a class with five people, the amount of peer assessments have been set to 3, the amount of overassessments has been set to zero (or in our logic, eliminated alltoghether), and also consider that the workshop is set up so that the start of the assessments period is the same as the start of the submissions period (i.e., set so that students can begin to do peer assessments as soon as they have submitted their work). Students A, B, C, and D are prompt and submit their work as soon as the workshop allows them to and then they continue to promptly do their peer assessments. This chart shows that:
Student A assesses B, C, D.
Student B assesses A, C, D.
Student C assesses A, B, D.
Student D assesses A, B, C.
Then along comes Student E who now submits his/her work, and on time. Because all four of the other students have done the three assessments that they are supposed to and no more, Student E has nobody to assess their work. Also, Student E cannot assess anybody else's work, because everybody else already has their three assessments.
The module works by creating fake, aka "hot" assessments with imposable grades set to a year in the future (both of which have potential problems of their own) to whatever student attempts to view the listing of assessments that they are to do. The number that it creates is essentially determined by the number of assessments that students should do, added to the amount that was set as overallocation, with all of that subtracted by the amount of assessments that they have (which includes those fake hot assessments.)
One proposal is to simply remove the offending if statement, and then change the workshop creation process so that the Teacher has to define a separate Assessments Only period that would begin after the Submissions Only period has ended. This would ensure that every student would have the proper amount of assessments to do, determined at random so that everyone gets their work assessed. However, this likely means that the overall time spend in a particular workshop would increase, along with the loss in flexibility.
What we propose to do is to duplicate this piece of logic into two identical pieces where one version would have the offending if statement removed. The first, and original piece of logic, would ensure that the students are given some, but not all, of the assessments to do immediately after they have submitted their own work. Then after the Submission period has closed, the logic would change to the other piece, this one with the same if statement removed. This would then fill in the remanding assessments, if any, that have not been given to a student. The only thing that we forsee is that you would need to have your End of Assessments time be longer that your End of Submissions time. However, this would not have to be very long, since only a very few students would have to do a few assessments. And, with the automatic e-mail messages, this should not be a large problem.
It's likely that there is a better solution, and we probably could come up with one. Unfortunately, a better solution would likely entail a massive code rewrite. (We'd much rather do it right than to hack together a solution that magically works.) However, that would greatly exceed the time available to spend on this project. Personally, I would consider a massive re-write a requirement for a future workshop module maintainer.
We are now asking for your opinion. Do you think that the proposed change would force you to change the way that you implement the workshop? Do you have a better solution in mind? What do you think is the best thing to do? We hope to send our changes upstream into the Moodle codebase, so your feedback is vital to us.
Thank You. (and sorry about the massive post)
After talking with The Professor (tm), we noticed that I made a mistake in my post. With the if statement removed, the later submitter would get the proper number of assessments to do, but that they would not get feedback on their work because everyone else has done their assessments.