I have been working on developing a module to facilitate the peer review of student work. The goal is to make it easier to use than the workshop module and to support the review of existing student work (ie assignments that have already been created/completed) or of teacher provided examples. Some of the features include:
* Support for review by rubric and/or by an uploaded annotated file
* Support for anonymity (or not)
* Reviews other modules (currently only the assignment module and resources)
* Automatic and manual assignment of reviewers to students (can copy the assignments from another peer review)
* Allows the teacher to look over and comment on each review a student receives and to additionally review the students work.
* Rubrics
* Based off of the quiz module (1.4) with the same setup
* Reusable at the criteria and rubric level
* Rubric questions allow for a level (excellent, good, poor, etc.), an exemplar (Work at this level has the following features ...) and an optional comment box for more detailed information.
* Usable by both teachers and students
It is still in the works, but I'm hoping to have it in working order and do some evaluation on it in the next few months.
By browsing through some of the forums, I can see that there has been some interest in peer review and that there has been other modules created for this purpose (though the one I found was a couple of years old). I'm curious if anyone has any comments or suggestions.
Scott
Scott, I would love to have a peer review. It is would be a great tool to allow students to think and respond to other's work and examples.
Please let me know when the module is ready.
Rhonda
I saw a combination of Rubrics and "divide a box of stars/points" under the team members on Ipeer http://ipeer.apsc.ubc.ca/ipeer_site/
Inspiration for your design?
Not directly, no. Our design came mostly from educational theory and from our survey of papers on peer review (mostly in Computer Science classes) and the accompanying systems. We found most of the papers had or mentioned a need for something rubric-like, but, if it existed, it was fairly primitive or very narrowly focused. It just seemed to be a need that was not being met.
Scott
Scott
I did not mean: was that your inspiration but could that give you new input 
Oops, misread your message. Thanks for the link.
Scott
Scott
HI,
I've just joined this forum and have been looking for a module for peer review. Is there a chance of a sneak preview?
I'm anxious to see how it works. I was considering using ipeer, but if a moodle alternative is available then I'll use that instead.
Any one tried ipeer? Is anyone trying to integrate ipeer or other similar programmes?
I've just joined this forum and have been looking for a module for peer review. Is there a chance of a sneak preview?
I'm anxious to see how it works. I was considering using ipeer, but if a moodle alternative is available then I'll use that instead.
Any one tried ipeer? Is anyone trying to integrate ipeer or other similar programmes?
Peter,
I'm looking into getting it set up on a public server so other people can play around with it as a teacher and as students. I'll let you know when that happens.
Scott
I'm looking into getting it set up on a public server so other people can play around with it as a teacher and as students. I'll let you know when that happens.
Scott
Peter and anyone else that is interested,
I've placed a demo copy up on a public server so people can play with my peer review module. If you are interested, email me (it's in my profile) and I'll send you the needed info. It is still in the development stages, so I know there are interface problems, but most of the functionality should be there and working.
Scott
I've placed a demo copy up on a public server so people can play with my peer review module. If you are interested, email me (it's in my profile) and I'll send you the needed info. It is still in the development stages, so I know there are interface problems, but most of the functionality should be there and working.
Scott
Is this code available anwhere?
We are about to start on a peer review project and it would be nice to finish something in progress rather than start from scratch.
TIA
Michael
We are about to start on a peer review project and it would be nice to finish something in progress rather than start from scratch.
TIA
Michael
I don't have any code, but I have been thinking about this for a while and I have some ideas.
I don't have much time now to elaborate (I'll do this later on if you are interested and there is a discussion).
Basically, after the assignment, essay question on a quiz or whatever form of task is submitted by the students, each student is presented with a set of answers/essays/products from other students. These answers/essays are anonymous and selected at random. The number of answers to be reviewed by each student is selected by the teacher.
This is optional but the student can also be presented with with a model answer prepared by the teacher together with a rubric. Then the student must evaluate the "answers" by their peers. Another option would be that, in addition, the student can evaluate his/her own "answer" as well, having seen the model answer and the rubric.
After all the essays/answers have been evaluated by the students, the teacher is presented with a grade report (which can borrow many things from the grade module you guys developed
) where one of the entries is the average grade obtained from all the evaluations received and crucially another piece of data is the standard deviation or some other statistical measure that allows the teacher to quickly visualize grades that are "unreliable" (because there is wide disagreement among the people who have evaluated that particular answer).
The teacher can intervene (especially in those grades where there is noticeable disagreement) by adding his/her own grade or even totally modifying the grade or she can decide to simply accept the average grade obtained via peer evaluation and make it the final grade.
An interesting option to add is the following: the teacher could tell the students that they will not only be evaluated for their answer but that they will also get a percentage of the grade derived from how good evaluators they are. There are two motivations for this. One is pedagogical: you show how well you understand an issue by giving the appropriate evaluation to an answer that discusses this particular issue. The second is practical: it is a measure to try to prevent "cheating" or more appropriately to ensure the maximum degree of fairness in evaluating. Students will try to be as fair as possible and they will be reluctant to be either too generous or too mean to their peers because they know that the teacher could take off points from their answers if they are not fair.
Again, a measure like standard deviation would signal whether the grade assigned by each student differs considerably from the grades assigned to the same answer by other students. This can make the life of the teacher easy because the teacher can decide to assign the points for being a good evaluator automatically: if there is no substantial difference between the grade given by a particular student and the grades given by other students (what is a "substantial difference" can be decided and when the numbers go over this threshold the figure for the standard deviation can be marked in a different color) then all the points are given, points are taken off according to how much the grade differs from the average of the other grades assigned by the other students; if there is a substantial difference between the grades assigned to a particular answer, the teacher can decide to look into that particular evaluation to see which evaluation was the most appropriate.
So, besides the good pedagogical contribution of peer evaluation, for classes that have a lot of students, this would save the teacher a lot of work. The teacher would, of course, always have the freedom to look into however many essays/answers and evaluations s/he wanted and intervene as much as s/he considers necessary.
I don't know whether I'm making a lot of sense. As I said, I'm in a bit of a hurry. I will clarify whatever is not clear later on if anyone has trouble understanding my basic proposal.
Josep M.
I don't have much time now to elaborate (I'll do this later on if you are interested and there is a discussion).
Basically, after the assignment, essay question on a quiz or whatever form of task is submitted by the students, each student is presented with a set of answers/essays/products from other students. These answers/essays are anonymous and selected at random. The number of answers to be reviewed by each student is selected by the teacher.
This is optional but the student can also be presented with with a model answer prepared by the teacher together with a rubric. Then the student must evaluate the "answers" by their peers. Another option would be that, in addition, the student can evaluate his/her own "answer" as well, having seen the model answer and the rubric.
After all the essays/answers have been evaluated by the students, the teacher is presented with a grade report (which can borrow many things from the grade module you guys developed
) where one of the entries is the average grade obtained from all the evaluations received and crucially another piece of data is the standard deviation or some other statistical measure that allows the teacher to quickly visualize grades that are "unreliable" (because there is wide disagreement among the people who have evaluated that particular answer). The teacher can intervene (especially in those grades where there is noticeable disagreement) by adding his/her own grade or even totally modifying the grade or she can decide to simply accept the average grade obtained via peer evaluation and make it the final grade.
An interesting option to add is the following: the teacher could tell the students that they will not only be evaluated for their answer but that they will also get a percentage of the grade derived from how good evaluators they are. There are two motivations for this. One is pedagogical: you show how well you understand an issue by giving the appropriate evaluation to an answer that discusses this particular issue. The second is practical: it is a measure to try to prevent "cheating" or more appropriately to ensure the maximum degree of fairness in evaluating. Students will try to be as fair as possible and they will be reluctant to be either too generous or too mean to their peers because they know that the teacher could take off points from their answers if they are not fair.
Again, a measure like standard deviation would signal whether the grade assigned by each student differs considerably from the grades assigned to the same answer by other students. This can make the life of the teacher easy because the teacher can decide to assign the points for being a good evaluator automatically: if there is no substantial difference between the grade given by a particular student and the grades given by other students (what is a "substantial difference" can be decided and when the numbers go over this threshold the figure for the standard deviation can be marked in a different color) then all the points are given, points are taken off according to how much the grade differs from the average of the other grades assigned by the other students; if there is a substantial difference between the grades assigned to a particular answer, the teacher can decide to look into that particular evaluation to see which evaluation was the most appropriate.
So, besides the good pedagogical contribution of peer evaluation, for classes that have a lot of students, this would save the teacher a lot of work. The teacher would, of course, always have the freedom to look into however many essays/answers and evaluations s/he wanted and intervene as much as s/he considers necessary.
I don't know whether I'm making a lot of sense. As I said, I'm in a bit of a hurry. I will clarify whatever is not clear later on if anyone has trouble understanding my basic proposal.
Josep M.
Joseph,
I'm also quite interested in this. I have looked at the CPR web site at http://cpr.molsci.ucla.edu . It looks like it's based on the same theory, but has some limitations that hosting your module would remove. Please keep me posted.
Joseph,
I'm also quite interested in this. I have looked at the CPR web site at http://cpr.molsci.ucla.edu . It looks like it's based on the same theory, but has some limitations that hosting your module would remove. Please keep me posted.
Scott,
Great to hear that you are working on a Peer Review tool for Moodle.
IS this available yet?
Great to hear that you are working on a Peer Review tool for Moodle.
IS this available yet?
jd
i need the peer rating system for my ongoing project. Can you please tell me whether it is available now or not ? Please its very urjent. Help me.
Hi:
I'm trying out your PeerReview module just now. Excellent work... thank you!
One question so far: the module limits students to two reviews of work submitted by other students. A bit limited for my purposes in writing courses-- more examples would be more useful, I think.
Can the module be set to force students to review more submissions than just two? Could the module permit the instructor to determine how many submissions students need to review?
I'm using the Peer Review Assignment Type, but am not seeing any of the submitted assignments in the Submissions Tab. It just indicates "Nothing to Display."
I know students have submitted assignments. Could this be caused by something that we don't have installed correctly. Or, is there a setting I'm missing?
Thanks for any help you can provide.