Are rubrics really a good thing? Upcoming webinar

Are rubrics really a good thing? Upcoming webinar

by Tim Hunt -
Number of replies: 13
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

As you probably know, Moodle includes Rubrics as an advanced grading method.They are said to be a way to increase the consistency of grading, and help the communication between teacher and student, but do they acutally do that?

There is a webinar coming up next week, which looks at some research about that. I just thought it might be of interest to people here.

Speaker photoThe multiple limitations of assessment criteria
Presenter: Emeritus Professor Sue Bloxham (University of Cumbria, UK)
Date and time: 7-8am GMT, Wednesday 4th November 2015.

This session will explore the use of assessment criteria by experienced markers. Sue will draw on a recent study which identified five factors in the use of criteria that contribute to marking inconsistency. The implications for fairness, standards and guidance to students will be discussed. Discussion will also take place on alternatives and enhancements to assessment criteria as typically used in higher education.

Sessions are hosted by Professor Geoffrey Crisp, Dean Learning and Teaching, RMIT University and Dr Mathew Hillier, Office of the Vice-Provost Learning and Teaching, Monash University, Australia.

Starting UTC 07:00am (5PM Eastern Australian standard) time for 1 hour.

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Are rubrics really a good thing? Upcoming webinar

by Mary Cooch -
Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

The last time I watched something suggested by you - if I recall correctly, a presentation on testing with quizzes by someone you knew -  it was really interesting - so I will try to watch this live, or if not then watch the recording soon afterwards. 

Average of ratings: -
In reply to Mary Cooch

Re: Are rubrics really a good thing? Upcoming webinar

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

That migth have been the recording of Joshua Bragg's MootUS15 talk:


Average of ratings: Useful (3)
In reply to Mary Cooch

Re: Are rubrics really a good thing? Upcoming webinar

by Mary Cooch -
Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

(off topic) Tim - no it was a woman, possibly colleague or someone working at the OU, a live presentation at a conference a year or two back. Long talk but worth watching.

Average of ratings: -
In reply to Mary Cooch

Re: Are rubrics really a good thing? Upcoming webinar

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

This one? http://surf.mediamission.nl/Mediasite/Play/193a63f439d540c89e16c24e73a3aa131d?catalog=256d42e5-b866-4751-a230-3e85dcb0f01a

"Computer marked assessment: Friend of foe?" keynote by Sally Jordan in 2014.

I know this is heading off topic, but these are all interesting talks, so I don't mind linking to them again.

Average of ratings: Useful (3)
In reply to Tim Hunt

Re: Are rubrics really a good thing? Upcoming webinar

by Mary Cooch -
Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

Yes big grin That was it -really good.

Average of ratings: -
In reply to Tim Hunt

Re: Are rubrics really a good thing? Upcoming webinar

by Don Hinkelman -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Thanks for the recommendations.  I just watched them and would vote them as the best two Moodle Presentations of 2015. That is because I am deeply immersed in assessment design and issues right now. Sally made a good point at the end about we have to solve technical issues of sharing questions.  It is so difficult and Moodle struggles with it.

Average of ratings: -
In reply to Tim Hunt

Re: Are rubrics really a good thing? Upcoming webinar

by Matt Bury -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Thanks for sharing Tim smile

Looks interesting. I've seen some terrible misuses of rubrics and it'll be great to get a well-researched overview of what usually goes wrong and, hopefully, how to get it right.

I checked out their Youtube channel and it looks like they upload video recordings of talks there. So if it's at some anti-social hour for anyone, they'll be able to watch a recording of it whenever they get round to uploading it.

Average of ratings: -
In reply to Matt Bury

Re: Are rubrics really a good thing? Upcoming webinar

by Don Hinkelman -
Picture of Particularly helpful Moodlers Picture of Plugin developers

I would like to hear Matt Bury talk about rubrics. At least, Matt, tell us about some of those terrible misuses of rubrics that you are aware of. For me, anything is better than the mysterious A, B, C or F. Numbers, too, have little me except for flashing to my best friends that I was the top. Rubrics introduce qualitative scales and can combine numbers so students know what the target is and whether they passed the minimum or achieved the best. Then again, I am curious to know the dangers, because I am always far too uncritical. smile

Average of ratings: -
In reply to Don Hinkelman

Re: Are rubrics really a good thing? Upcoming webinar

by Matt Bury -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Hi Don,

Re: "...tell us about some of those terrible misuses of rubrics that you are aware of." -- Top of my list would be template rubrics that teachers take and use unchanged or adapted slightly and use them for any number of different assignments/tasks. When rubrics are generic and vague, they're of no use to anyone and just as subjective as % and letter grades. Ideally, rubrics should be as specific and pragmatic as possible so that they are immediately meaningful to the task at hand and, above all, descriptive rather than prescriptive, i.e. not a list of instructions; that's what the assignment/task instructions are for.

As we both come from a second/foreign language learning background, a good example of misuse of template rubrics is the Council of Europe's Common European Framework of Reference for Languages (CEFR). They came up with some very general descriptors for each of the 6 levels (and subsequently 5 more in-between levels) which they intended to be used as a starting point for teachers and curriculum developers to adapt and make them specific to contexts and tasks (See an example here: http://matbury.com/html/cefrl/pdf/cefrl_can_do_b1.pdf ). However, once "released into the wild," they were used verbatim regardless of tasks, contexts, or learners' needs and, in some cases, even used as end of course report cards to show learners' progress.

Used as they are in this way turns them into little more than a "criteria check-list" with a 3-point Likert scale next to each item.

Does that make sense? Ring any bells from what you've seen in your experience?

Average of ratings: -
In reply to Matt Bury

Re: Are rubrics really a good thing? Upcoming webinar

by Don Hinkelman -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Thanks, Matt. I was hoping you would say this because it matches my experience:

When rubrics are generic and vague, they're of no use to anyone and just as subjective as % and letter grades. Ideally, rubrics should be as specific and pragmatic as possible so that they are immediately meaningful to the task at hand and, above all, descriptive rather than prescriptive, i.e. not a list of instructions; that's what the assignment/task instructions are for.

I just watched the videos recommended by Tim and that also influenced me in the complexity of homework, quizzing, testing and assessments.  Just to give a concrete example of what I feel is a specific rubric designed for a specific task in speech communication, here is one example done in Moodle.  So my answer to the opening question, "Are rubrics really a good thing?", is "Yes, if they are specific and part of a systematic design of open practice, timed practice with a mix of constructed/selected question items, and finally performance-based video assessment, all with appropriately timed feedback. (That performance point was not in the videos, but a topic of a recent talk I did).
speech rubric

Note: this rubric is for *both* learners and teachers, where peer/self/teacher/class assessments are all combined.

Average of ratings: Useful (2)
In reply to Don Hinkelman

Re: Are rubrics really a good thing? Upcoming webinar

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Agree with everything Don, rubrics can be useful but only if specifically directed at each task/assignment/presentation/etc. Been using IB standardised (or should that be "bastardised") rubric given me and told "This is it, the rubric we use for Science". Wow! I been spoken to from ON HIGH so how dare I speak in argument? But to my folly, I also argued assessment for learning/understanding, not memory... gawd you think I made me unpopular?   

Average of ratings: Useful (1)
In reply to Colin Fraser

Re: Are rubrics really a good thing? Upcoming webinar

by Matt Bury -
Picture of Particularly helpful Moodlers Picture of Plugin developers

The webinar has been posted on youtube:


Average of ratings: Useful (3)
In reply to Tim Hunt

Re: Are rubrics really a good thing? Upcoming webinar

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
I registered for the webinar, or hope so, it was a long procedure, and being greeted by this! Hope there'll be an off-line version.
;-(
Attachment blackboard_collaborate.png
Average of ratings: -