Two years ago I met Martin D. at Heidelberg during a nice german Moodlemoot (from which the we spaniards took some ideas for our moots), to discuss wiki 2.0 (back then!).
One of the topics was Grading on Wikis. We also debated this topic with skodak when he visited UPC.
Now that I the Wiki 2.0 looks stable enough to build "fancy" features upon here are my philosophy principles and requirements for the develoment that is starting as soon as Wiki 2.0 hits the moodle head cvs.
---
Its not obvious and easy to define how to grade a Wiki activity. What is clear is that there are as many ways to do it as teachers willing to. So everithing I describe needs to be optional and customizable "a la Moodle" (As pigui says in every meeting "use the moodle built in scales and don't invent new things")
---
We have two different sets of "things" to evaluate:
- Pages. And its evolution. Its what I will call "page rating" and "page progress rating "
- Grades. Wich are related to he work done for one student, and how does THE TEACHER consider that should be evaluated (and bound to the moodle grading engine... and all its complexity)
Page ratings and page progress activity.
--------------------------------
Who can rate a page? Everyone (peer review) if the teacher allows it. In my opinions this is the way to go, but the teacher has to enable it and allow the rating access to the students.
The page rating is somehow related to all the authors of the page, that can be found in the page history.
One user rating of a page, can change over time. In this case we can keep track of the "page progress" also related to the editors of the affected versions. We can enter these information when we change our mind or in the page-version compare view.
Ratings are bound to a scale, but also some fedback can be introduced. In this case... a coment could/should be created:
USER rated this page with 3* and says that "yep this page rocks!"
For instance
Grading
------
The teacher is the one who grades. So he needs a special page where all the posible information on pages, activity and ratings can be reported to him in different aggregated ways. So he can safelly grade each student and send the grade to the gradebook.
Comming soon screenshots and maybe videos of use cases.... feedback and chocolate is most welcome.
Ludo
About grades , grading on wiki 2.0 ... philosophy and requirements of ongoing development
by Ludo (Marc Alier) -
Number of replies: 2
In reply to Ludo (Marc Alier)
Re: About grades , grading on wiki 2.0 ... philosophy and requirements of ongoing development
by Xavier de Pedro -
Hi Ludo, and thanks for bringing this topic to the list at this (still) development stage.
I would suggest two things:
(1) that somehow the "type of contribution" concept (i.e., "Adding information", "summarizing information", "formulating new hypothesis", "improving markup", ...) is added as a multiple select box to the wiki edit interface.
(2) And that the size (b, kb, ...) of the new contribution is recorded.
This way, basic information about "quality" and "quantity" of information can be reviewed later on by faculty, provided that the report tool allows that in the future.
So that we can keep track of student activity (single students and average per group): not only the number of clicks (page views, page edits, etc.), but the quantity and quality of that information added or deleted by each student/group of students.
As you know,
, at University of Barcelona we did some research about this field, and we found out that this type of information is EXTREMELY useful for teachers before attending a tutoriship session with students (with individual students or in groups). Of course, some short training & feedback has to given to students in the first weeks so that they know what the "type of contributions" are for, and how to select the right ones, etc. (the faculty should be able to edit the records of those "type of contributions" to avoid misuse and pollution in the database of actions, which will be used later on by him/her and other faculty to provide feedback, grade students, etc.)
---
And on another stage, that information about "type of contributions" should be linkable to the "outcomes" that we expect the students to acquire with the course activities. The goal for this is two fold:
a) we promote that students see/remember the outcomes that we want them to work/acquire through the learning activities (promoting experiential - reflective learning is social-constructivist scenarios - their mates see also how their mates indicate what their are doing, so they learn from this metacognitive process from themselves and their mates)
b) we help (allow/ease...) faculty to assess student activity along the course, and not just at the end when they have the final version of the wiki finished/delivered at the assignment deadline. This helps single teachers to review in short what has happenend (in order of magnitude) for a given time frame with single students or whole group/s, looking and some summary graphs/tables. And students have the change to report better who where they working with when they submitted something to the wiki (or to the forum, spreadsheet, etc...)
Once this is implemented in Moodle, if others see it interesting for their use cases with wikis inside educational scenarios (opinions?), I would expect any automatic grading at the gradebook, but just a new column at the gradebook, and yes, some automatic feeding of a new table with students in raws, and type of contributions in columns, with the size of the change as value in the cells, for a given time frame selected by the theacher. With that, we can let every teacher convert the activity of students into a grade for the gradebook.
Does it make sense for you?
Ok, enough for my suggestion of improvements (from the perspective of a faculty not using Moodle yet with my students when I need to use wikis for real).
I wish someday I can use also Moodle for that (without having to clikc in so many places to have a broad idea of what has been happening with the activity of my students).
For those interested, more details and examples of graphs/tables of summary data of "quality and quantity" of contributions by students (and groups) using some other free software tools, can be read at:
* "New Method Using Wikis and Forums to Evaluate Individual Contributions in Cooperative Work while Promoting Experiential Learning: Results from Preliminary Experience" X. de Pedro. WikiSym’07 October 21–23, 2007, Montréal, Québec, Canada. http://www.wikisym.org/ws2007/_publish/dePetro_WikiSym2007_ContributionsInCooperativeWork.pdf
Alternatively, there is a longer and more detailed report (In Spanish):
* “Estimulación y evaluación del aprendizaje 'experiencial-reflexivo' del alumnado mediante la formulación explícita del tipo de contribuciones”. Grupo de trabajo sobre Aprendizaje Reflexivo (ICE-UB 2006). http://www.ub.edu/gclub/dl55
Hope this helps
Xavi
I would suggest two things:
(1) that somehow the "type of contribution" concept (i.e., "Adding information", "summarizing information", "formulating new hypothesis", "improving markup", ...) is added as a multiple select box to the wiki edit interface.
(2) And that the size (b, kb, ...) of the new contribution is recorded.
This way, basic information about "quality" and "quantity" of information can be reviewed later on by faculty, provided that the report tool allows that in the future.
So that we can keep track of student activity (single students and average per group): not only the number of clicks (page views, page edits, etc.), but the quantity and quality of that information added or deleted by each student/group of students.
As you know,
---
And on another stage, that information about "type of contributions" should be linkable to the "outcomes" that we expect the students to acquire with the course activities. The goal for this is two fold:
a) we promote that students see/remember the outcomes that we want them to work/acquire through the learning activities (promoting experiential - reflective learning is social-constructivist scenarios - their mates see also how their mates indicate what their are doing, so they learn from this metacognitive process from themselves and their mates)
b) we help (allow/ease...) faculty to assess student activity along the course, and not just at the end when they have the final version of the wiki finished/delivered at the assignment deadline. This helps single teachers to review in short what has happenend (in order of magnitude) for a given time frame with single students or whole group/s, looking and some summary graphs/tables. And students have the change to report better who where they working with when they submitted something to the wiki (or to the forum, spreadsheet, etc...)
Once this is implemented in Moodle, if others see it interesting for their use cases with wikis inside educational scenarios (opinions?), I would expect any automatic grading at the gradebook, but just a new column at the gradebook, and yes, some automatic feeding of a new table with students in raws, and type of contributions in columns, with the size of the change as value in the cells, for a given time frame selected by the theacher. With that, we can let every teacher convert the activity of students into a grade for the gradebook.
Does it make sense for you?
Ok, enough for my suggestion of improvements (from the perspective of a faculty not using Moodle yet with my students when I need to use wikis for real).
I wish someday I can use also Moodle for that (without having to clikc in so many places to have a broad idea of what has been happening with the activity of my students).
For those interested, more details and examples of graphs/tables of summary data of "quality and quantity" of contributions by students (and groups) using some other free software tools, can be read at:
* "New Method Using Wikis and Forums to Evaluate Individual Contributions in Cooperative Work while Promoting Experiential Learning: Results from Preliminary Experience" X. de Pedro. WikiSym’07 October 21–23, 2007, Montréal, Québec, Canada. http://www.wikisym.org/ws2007/_publish/dePetro_WikiSym2007_ContributionsInCooperativeWork.pdf
Alternatively, there is a longer and more detailed report (In Spanish):
* “Estimulación y evaluación del aprendizaje 'experiencial-reflexivo' del alumnado mediante la formulación explícita del tipo de contribuciones”. Grupo de trabajo sobre Aprendizaje Reflexivo (ICE-UB 2006). http://www.ub.edu/gclub/dl55
Hope this helps
Xavi
In reply to Xavier de Pedro
Re: About grades , grading on wiki 2.0 ... philosophy and requirements of ongoing development
by Ludo (Marc Alier) -
Thanks for our insights Xavier,
about 1: This could be gathered thought "a summary field" bellow the edit box and assigned to each page edit. so each contributor can assert what kind of participation is going on. also a good idea for reflection on what the heck I'm doing right now. But from the data gathering user interface I see it very difficult to do it peer rewies without cluttering too much the UI.
about 2: all this information will be gathered and presented to the teacher in the reports.
Maybe we could devise ome kind of balanced scorecard for the student so (s)he can know at any time his overal participation scores... (we can even think of giving badges away, a la foursquare, but this will have to wait for moodle 3.0
)
I will contact you son to have a work session live!
Cheers Xavi!
about 1: This could be gathered thought "a summary field" bellow the edit box and assigned to each page edit. so each contributor can assert what kind of participation is going on. also a good idea for reflection on what the heck I'm doing right now. But from the data gathering user interface I see it very difficult to do it peer rewies without cluttering too much the UI.
about 2: all this information will be gathered and presented to the teacher in the reports.
Maybe we could devise ome kind of balanced scorecard for the student so (s)he can know at any time his overal participation scores... (we can even think of giving badges away, a la foursquare, but this will have to wait for moodle 3.0
I will contact you son to have a work session live!
Cheers Xavi!