As you probably know, Moodle includes Rubrics as an advanced grading method.They are said to be a way to increase the consistency of grading, and help the communication between teacher and student, but do they acutally do that?
There is a webinar coming up next week, which looks at some research about that. I just thought it might be of interest to people here.
The multiple limitations of
Presenter: Emeritus Professor Sue Bloxham (University of Cumbria, UK)
Date and time: 7-8am GMT, Wednesday 4th November 2015.
This session will explore the use of assessment criteria by experienced markers. Sue will draw on a recent study which identified five factors in the use of criteria that contribute to marking inconsistency. The implications for fairness, standards and guidance to students will be discussed. Discussion will also take place on alternatives and enhancements to assessment criteria as typically used in higher education.
Sessions are hosted by Professor Geoffrey Crisp, Dean Learning and Teaching, RMIT University and Dr Mathew Hillier, Office of the Vice-Provost Learning and Teaching, Monash University, Australia.
Starting UTC 07:00am (5PM Eastern Australian standard) time for 1 hour.
"Computer marked assessment: Friend of foe?" keynote by Sally Jordan in 2014.
I know this is heading off topic, but these are all interesting talks, so I don't mind linking to them again.
Thanks for the recommendations. I just watched them and would vote them as the best two Moodle Presentations of 2015. That is because I am deeply immersed in assessment design and issues right now. Sally made a good point at the end about we have to solve technical issues of sharing questions. It is so difficult and Moodle struggles with it.
Thanks for sharing Tim
Looks interesting. I've seen some terrible misuses of rubrics and it'll be great to get a well-researched overview of what usually goes wrong and, hopefully, how to get it right.
I checked out their Youtube channel and it looks like they upload video recordings of talks there. So if it's at some anti-social hour for anyone, they'll be able to watch a recording of it whenever they get round to uploading it.
I would like to hear Matt Bury talk about rubrics. At least, Matt, tell us about some of those terrible misuses of rubrics that you are aware of. For me, anything is better than the mysterious A, B, C or F. Numbers, too, have little me except for flashing to my best friends that I was the top. Rubrics introduce qualitative scales and can combine numbers so students know what the target is and whether they passed the minimum or achieved the best. Then again, I am curious to know the dangers, because I am always far too uncritical.
Re: "...tell us about some of those terrible misuses of rubrics that you are aware of." -- Top of my list would be template rubrics that teachers take and use unchanged or adapted slightly and use them for any number of different assignments/tasks. When rubrics are generic and vague, they're of no use to anyone and just as subjective as % and letter grades. Ideally, rubrics should be as specific and pragmatic as possible so that they are immediately meaningful to the task at hand and, above all, descriptive rather than prescriptive, i.e. not a list of instructions; that's what the assignment/task instructions are for.
As we both come from a second/foreign language learning background, a good example of misuse of template rubrics is the Council of Europe's Common European Framework of Reference for Languages (CEFR). They came up with some very general descriptors for each of the 6 levels (and subsequently 5 more in-between levels) which they intended to be used as a starting point for teachers and curriculum developers to adapt and make them specific to contexts and tasks (See an example here: http://matbury.com/html/cefrl/pdf/cefrl_can_do_b1.pdf ). However, once "released into the wild," they were used verbatim regardless of tasks, contexts, or learners' needs and, in some cases, even used as end of course report cards to show learners' progress.
Used as they are in this way turns them into little more than a "criteria check-list" with a 3-point Likert scale next to each item.
Does that make sense? Ring any bells from what you've seen in your experience?
Thanks, Matt. I was hoping you would say this because it matches my experience:
When rubrics are generic and vague, they're of no use to anyone and just as subjective as % and letter grades. Ideally, rubrics should be as specific and pragmatic as possible so that they are immediately meaningful to the task at hand and, above all, descriptive rather than prescriptive, i.e. not a list of instructions; that's what the assignment/task instructions are for.
I just watched the videos recommended by Tim and that also influenced me in the complexity of homework, quizzing, testing and assessments. Just to give a concrete example of what I feel is a specific rubric designed for a specific task in speech communication, here is one example done in Moodle. So my answer to the opening question, "Are rubrics really a good thing?", is "Yes, if they are specific and part of a systematic design of open practice, timed practice with a mix of constructed/selected question items, and finally performance-based video assessment, all with appropriately timed feedback. (That performance point was not in the videos, but a topic of a recent talk I did).
Note: this rubric is for *both* learners and teachers, where peer/self/teacher/class assessments are all combined.
Agree with everything Don, rubrics can be useful but only if specifically directed at each task/assignment/presentation/etc. Been using IB standardised (or should that be "bastardised") rubric given me and told "This is it, the rubric we use for Science". Wow! I been spoken to from ON HIGH so how dare I speak in argument? But to my folly, I also argued assessment for learning/understanding, not memory... gawd you think I made me unpopular?