Criteria Referenced Assessment

Criteria Referenced Assessment

by Stephen Wilkinson -
Number of replies: 8

I'm not sure if this is the right forum to ask (perhaps this should go into general development or wish-lists) but here goes...

We use quite a different type of assessment system on our courses. Students complete assignments, activities, practical-tasks, research etc. and then produce portfolios of evidence. These portfolios are then assessed against criteria, leading to grades against various strands of competencies.

We have been using Moodle for some years, and do make use of the assignments and quizzes to assess students. We also assess students use of the forums, choices, lessons etc.

At the moment we cannot use the Moodle gradebook functionality as we would wish. We have created a new scale for our assessment and can then use this to mark individual assignments. But the gradebook cannot conflate these grades into our strands as it makes use of numbers and we do now. (Neither can we just convert our grades to numbers - criteria references assessment does not work like that!)

We want a more flexible gradebook system in which you can create complex rules to produce profiles of attainment. For example, if a student gets grades AABACCDCAAA against 10 assignments we want to be able to produce a profile that, for example, give the student an overall assessment of A (for strand 1, based on assignments 1, 2, 8 and 9) and C (for Strand 2, based on assignments 5, 6 and 7).

We also want to be able to grade other moodle activities. At the moment an assignment has to be a piece of work that is typed or uploaded or something completed offline. Assessment can be based on so much more.

Just some initial thoughts to get ideas going I suppose.

Any responses??

In reply to Stephen Wilkinson

Re: Criteria Referenced Assessment

by Matt Gibson -
Sounds good. I have something similar with essays marked on two scales - knowledge & understanding and analysis & evaluation, but it's hard to see this in the gradebook or even the essay itself. The workshop does something like this with rubrics, but it's not cross-site. I'd like to look in the gradebook and see striaght off who is weak on analysis and would benefit from extra help.

Additionally, it would be great to have a view where the deadlines are displayed separately. I'm just doing our academic reviews and I'm having to make a separate spreadsheet to see who is late handing submitting stuff.
In reply to Matt Gibson

Re: Criteria Referenced Assessment

by Steve Power -
We would like this here too as this is a major reason we are not using assignments. We need in fact to mark them on 3 or 4 criteria or so I am told!
In reply to Matt Gibson

Re: Criteria Referenced Assessment

by Stephen Wilkinson -

With regards to deadlines, I've never quite agreed with how the submitted field works for offline assignments.

At the moment this gets timestamped when the tutor grades and/or sends feedback.

What I would like to have is a way for the tutor to timestamp one/many/group/class so that they could check off and confirm that assignments have been submitted. The grading and feedback could then be submitted later - following the same model as online and uploaded assignments.

In reply to Stephen Wilkinson

Re: Criteria Referenced Assessment

by Lael ... -
Something like this would be great... I would love to see 'outcomes' or indicators being able to be mapped against each activity, question in a quiz, lesson etc to give some form of calculated feedback on each outcome.

This is not the same as criterion referencing, but if there were options for an application of scales (or criteria) to each question (and then that went through to the gradebook), I think that would achieve what you are talking about.

(Though you might use the term strand in place of outcomes... or maybe not, perhaps if outcomes could be grouped and their results somehow combined to form results for categories or strands...)

I love the opportunities that this could provide!

Lael
In reply to Lael ...

Re: Criteria Referenced Assessment

by Stephen Wilkinson -

I think it would mean a lot of re-programming, as almost every type of activity might need to be able to be assessed, and assessed in different ways.

At the moment I don't really see the assignment module as an assignment module. It is much more like an upload/online text module that is assessed. Sometime we want students to upload stuff by a certain deadline but we do not assess it - this would be the same module with the assessment switched off.

Along the same lines, I would like to able to switch on assessment for Chat, Choice, Database, Forum, Glossary, Quiz, Lesson, Questionnaire, Wiki and Workshop.

The gradebook would need to be able to work with user-defined schemas of scales and assessment criteria. I'll try to explain...

At the moment I can define a scale, and relate it to marks. I can also create activities.

I want to be able to attach assessment criteria to activities. These could be complicated criteria that the teacher would have to use to assess and record marks, or just pass/fail, against each of the assessed activities. However, they might also be simple criteria. For example, achieving more than 50% in a quiz might give a grade B; posting a reply on a forum might tick an evidence box for "online participation". In simple cases like this, it would be useful if the teacher did not have to manually enter the assessment.

The gradebook would need to be able to conflate and convert these marks, using the uder-defined schema, into final grades or even one final grade (dependent on the schema used).

I hope this makes some sort of sense!

Is this possible? Is anyone working on anything like this? Please...!

In reply to Stephen Wilkinson

Re: Criteria Referenced Assessment

by Matt Gibson -
That sounds like something I was thinking of - having the ability to define assignments as 'go online and read X' or 'post twice in X foum'.

The logs already have this data, but finding out whether it's happened is laborious. Being able to get evidence checked off automatically is a great idea.
In reply to Stephen Wilkinson

Re: Criteria Referenced Assessment

by Annick F -

"I want to be able to attach assessment criteria to activities. These could be complicated criteria that the teacher would have to use to assess and record marks, or just pass/fail, against each of the assessed activities."

We would also like to see this kind of feature available in Moodle.

Is this something that might already be in development?

Many Thanks.

In reply to Stephen Wilkinson

Re: Criteria Referenced Assessment

by Don Hinkelman -
Picture of Particularly helpful Moodlers Picture of Plugin developers
Hi Stephen,

Are you looking for the kind of assessment that is in the Project Module? If it is, then we can propose this as a cross-module standard.
http://moodle.org/mod/forum/discuss.php?d=69803

I also raised a similar question in the new Gradebook topic (look 3/4 down). I called them "Sets of Assessment Scales/Criteria". I believe the word "Rubrics" also applies.
http://moodle.org/mod/forum/discuss.php?d=69223