MoodleRoom's proposed Outcomes changes

Re: MoodleRoom's proposed Outcomes changes

by Martin Matthews -
Number of replies: 6

I freely admit that much of the Moodle-speak in what I have looked at here goes straight over my head and that might be why others tend not to comment.  However here are some comments from a purely educational practitioner viewpoint:

  1. it would be good to see a less USA-centric description toencourage development to be flexible.  I realise it is easier to speak of what you know but please do not neglect the non-US market as there are many of us!
  2. a big part of my concerns regarding outcomes (as they are now) is the viewing of those and their grading and a better gradebook view of outcomes would help a lot (but see also 3)
  3. I use outcomes all the time and my use of them may refelect that of others.  I do not care whether students pass an assignment/quiz/activity at all!  What matters is if they hit the outcomes.  Admittedly the one normally implies the other but it is a very different emphasis.  It makes no difference to students (in terms of qualification) how good their work is just whether they hit the outcomes.  This viewpoint should be reflected in an ability to track outcomes rather than grades.  Activities might be just marked complete/not complete (or percentage) as currently but the outcomes can be seen clearly as met or not met.
  4. results should be clearly visible at least course-wide so outcomes in the whole unit/moduel can be seen in one place.  Ideally it would also be possible for a student to see outcomes across Moodle courses as well (for those of us who use a separate course for each unit/module ina  qualification).  I am also aware of some units/modules which are taught by more than one person and I guess some might use separate Moodle courses and yet the outcomes are only relevant as a whole.  I suspect you would not want to include this in this particular project as it would presumably be best done as a separate plug-in.
  5. Have you considered what will happen if one outcome is met across multiple Moodle activities?  Often an outcome might say something like "Show examples of dealing with clients in ..." or it might say "Design, create and test ..." and again that naturally exists as three activities or more.  This is not hypothetical as I have already had to split outcomes including one into 8 separate outcomes (even though it is one official criterion).   For me the answer is simple - each outcome should have a configurable number of "mets" and for it to be completed it would have to be met in that many activities.  Students complete an activity and the outcome can be shown as 1/3 met or a percentage.  This simplistic approach may well not work for others so a one-to-many use of sub-outcomes might be unavoidable

Overall this seems to me to be a potential monster so well done for tackling it!

I am aware of PHP (but not OO PHP) and databases so if I can be of any use in opinion or testing please let me know.  The way Moodle works behind the scenes is something I have not looked at though so I may not be much use!  Hope this helps.

Average of ratings: Useful (1)
In reply to Martin Matthews

Re: MoodleRoom's proposed Outcomes changes

by Davo Smith -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Just one more point to throw in to this discussion about outcomes.

When I was still teaching, I tried a few times to use outcomes to record marks for BTEC qualifications. These work by having a number of Pass (P1, P2, P3, etc.), Merit (M1, M2, etc.) and Distinction (D1, D2, etc.) marks across as many assignments as the college wants to provide for each unit. Each mark can appear in more than one assignmet (theoretically anyway, it rarely happens in practice), although each student can only gain each mark (P1, P2, M1, etc.) once for a unit.

To gain a 'pass' for a unit, the student needs all the pass marks for that unit. All the pass and merit marks are needed for a 'merit'. All the marks are required for a distinction. 

I found that using outcomes for this was very unwieldy, as you had a large number of 'outcomes' in each course (especially if each course contained multiple units, in which case they ended up as U5P1, U5P2, etc. for Unit 5 P1). Each outcome also only needed a grade of 'Yes' or 'No'. There was also no sensible way to gather these grades into an overall pass/merit/distinction for a unit.

It may well be that this use case is completely outside of anything that outcomes will ever sensibly address (and maybe there needs to be some different mechanism to support such 'exotic' marking schemes - some sort of 'yes / no' criteria, rather than graded outcomes).

Average of ratings: -
In reply to Davo Smith

Re: MoodleRoom's proposed Outcomes changes

by Martin Matthews -

That's exactly how I use it Davo except that we only have one BTEC unit per Moodle course so it is more manageable.  However, it is still difficult to spot a final grade in a unit so your point is a good one. 

The point you made in passing about potentially a criterion appearing in more than one assignment is also a good one and might arise in at least two possible situations and I only thought of one originally:

  1. when a criterion cannot be met in one assignment as I mentioned in my earlier post
  2. where a student might be given more than one opportunity at a criterion

The second may not normally occur but in the UK I see it in the near future.  We are moving towards each student having an individual learning plan which meets their particular needs.  This, in theory, would mean that a student could choose between a range of tasks to meet a criterion.  Any one of those outcomes met would mean they had also met the criterion.

Hope that made sense my current group is a bit loud!

Average of ratings: Useful (1)
In reply to Martin Matthews

Re: MoodleRoom's proposed Outcomes changes

by Kris Stokking -

Martin/Davo - I wanted to follow up on the conversation around BTEC support.  Marcus Green created a BTEC advanced grading method (discussed here) that could be used in combination with BTEC Outcomes.  In this case, the relevant criteria for BTEC would be created within the advanced grading form and mapped to the corresponding Outcome.  The mapping could occur in more than one activity, which would handle when a student is given more than one attempt to satisfy a criterion.  A custom  BTEC recommendation plugin could assist instructors in marking the student's achivement of outcomes in the Instructor Student Completion Marking.  Once met, the student would carry the outcome achievement site-wide, and the data could be used for custom BTEC reporting if the out-of-box reports didn't satisfy your requirements.  The approach would only work for modules that support Advanced Grading Methods (currently only assignment), but certainly could be more useful as adoption for advanced grading increases.  

Average of ratings: -
In reply to Davo Smith

Re: MoodleRoom's proposed Outcomes changes

by Kris Stokking -

Davo, the Outcomes system allows for a simple identification of Met or Not Met (yes/no) for a student against an outcome.  We've had overwhelming feedback that this simplification is needed instead of the current implementation that is tied to the gradebook and based on scales. In addition, the algorithm for calculation is left completely open ended - we'll develop our own implementation using a streaks algorithm, but that's just an example of one use case.  I'm not sure how you're using a Unit - is that equivalent to a course section (i.e. topic)?

If I understand you correctly, I would use a Rubric that contains a criterion associated for each of the relevant unit marks.  Each criterion would be associated with a P, M and D outcome.  The instructor would mark off whether the student received a P, M or D, and the algorithm would mark off each outcome as Met or Unmet depending on the algorithm.  It could do this even if the Outcome was used in different assignments or even courses (depending on how Units apply). 

This might give you extra information in an Outcomes report as it would show each Pass, Merit and Distinction outcome as either Met or Unmet instead of showing each outcome as Pass, Merit or Distinction.  However, you could probably customize the report in some way to get the information you need.

Average of ratings: -
In reply to Martin Matthews

Re: MoodleRoom's proposed Outcomes changes

by Kris Stokking -

Martin, in regards to #5, our intention is to house the algorithm for determining whether an Outcome should be marked as Met or Not Met within a plugin system, giving developers complete control over how that should be computed.  The algorithm would have access to everywhere the Outcome was used, including different different activities or courses.

Average of ratings: -
In reply to Martin Matthews

Re: MoodleRoom's proposed Outcomes changes

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Martin "it would be good to see a less USA-centric description to encourage development to be flexible.  I realise it is easier to speak of what you know but please do not neglect the non-US market as there are many of us!"

+1. Except I do not like the word "market" and would rather use "users".

Joseph

Average of ratings: Useful (3)