Quiz

Marks, accuracy and CBM

 
 
Picture of David Korfhage
Marks, accuracy and CBM
 

I'm experimenting with Moodle quizzes as formative assessment, and trying CBM.  Based on a blog post I read (and also student reactions), I'm interested in having students see their accuracy (% correct) as well as their CBM adjusted score.  I think they would find that less troubling as well as more useful (e.g., a big gap between their accuracy and their adjusted score would tell them something.  

I've read that there have been proposals for implement this.  Has this been fully developed yet?  Is there a way to do this?

Thanks.

 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: Marks, accuracy and CBM
Group DevelopersGroup Documentation writersGroup Particularly helpful Moodlers

You are right. That functionality should be there. Tony Gardiner-Medwin has some code that does it, which you can get from http://www.tmedwin.net/cbm/moodle/. That is not suitable for including in the standard Moodle release, and I have not finished reworkign it. See MDL-32188 for details of that.

 
Average of ratings: -
Picture of David Korfhage
Re: Marks, accuracy and CBM
 

Thanks.  Just to clarify--that functionality will probably be there at some point, but it's not currently something I can do.  Do I understand that correctly?

And are there any workarounds?

 

 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: Marks, accuracy and CBM
Group DevelopersGroup Documentation writersGroup Particularly helpful Moodlers

Correct, it will happen some time.

The only work-around right now is to install Tony's code, which involved patching core Moodle.

 
Average of ratings: -
argm
Re: Marks, accuracy and CBM
Group Particularly helpful Moodlers

Quite a lot of people have downloaded these patches for 1.9, 2.0, 2.2, 2.3, 2.4 from http://www.tmedwin.net/cbm/moodle/ and I haven't had reports of any problems using them. They are pretty innocuous because they don't alter the database, and are easily removed if preferred for any reason. I find it hard to work with current Moodle coding conventions, so would welcome anyone interested in working with Tim and myself to get these patches into core Moodle. Meanwhile the CBM facility at www.ucl.ac.uk/lapt is a good deal more sophisticated in many ways and is open to other institutions to use for student self-tests.   

 
Average of ratings:Useful (1)
argm
Re: Marks, accuracy and CBM
Group Particularly helpful Moodlers

I've put up new CBM files for use with the new Moodle versions 2.2.8 and 2.3.5. There were only small changes in relevant core files since 2.2.6, 2.3.3 and none in 2.4.2 compared to 2.4.1. Use the 2.4.1 download if you have 2.4.2. I haven't changed the functionality of the mods. See http://tmedwin.net/cbm/moodle/ .

 
Average of ratings: -
argm
CBM Files for 2.5
Group Particularly helpful Moodlers

Files with improved CBM functionality for Moodle 2.4, 2.5 (rcl) are now available at my download site  http://tmedwin.net/cbm/moodle/ . There are no recent changes in the functionality of these mods, just updates to incorporate small changes made elsewhere in those core files that need to be replaced. Please let me know (ucgbarg at ucl.ac.uk) if there are any issues or queries. I don't automatically receive emails from this forum. TGM

 
Average of ratings: -
argm
CBM files for 2.5.1, 2.4.5, 2.3.8, 2.2.11, 1.19.19
Group Particularly helpful Moodlers

New patch files for improved CBM functionality in today's Moodle releases are now available at my download/demo site  http://tmedwin.net/cbm/moodle/ . There are a few changes merging minor differences in the core files that you need to replace. No adjustments were needed for the newest versions of Moodle 2.2 or 1.9, so the patches remain the same. 

I recently circulated those who have downloaded these patches in the last couple of years, asking if anyone has had problems. None were reported. If you know of any problems, even minor ones, please report them to me, ucgbarg at ucl.ac.uk, and if you like, raise them here. Some of the enhancements of functionality are substantial and without them CBM in Moodle may unfortunately seem rather obscure.

 
Average of ratings: -
argm
Re: CBM files for Moodle 2.5.2, 2.4.6, 2.3.9
Group Particularly helpful Moodlers

In Moodle 2.5.2 and 2.4.6 (just released) there are minor changes in 3 of the core files that need to be replaced for proper CBM functionality. The required patch files for these versions are  now available at  http://tmedwin.net/cbm/moodle/ . The patches for 2.3.8 are still valid for 2.3.9 since there are no changes in the relevant files. Let me know of any problems using these patches. TGM

 

 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: CBM files for Moodle 2.5.2, 2.4.6, 2.3.9
Group DevelopersGroup Documentation writersGroup Particularly helpful Moodlers

OK, so I finally made some time to work on MDL-32188, in the hope of getting it done for Moodle 2.6 (which means that it needs to be done this week.)

I just made a long comment there. If you are interested in this, any help you can give witht the outstanding issues would be apprectiated.

 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: CBM files for Moodle 2.5.2, 2.4.6, 2.3.9
Group DevelopersGroup Documentation writersGroup Particularly helpful Moodlers

I am making progress. I think the code on MDL-32188 is basically done, and I plan to test it a bit more tomrrow, and then submit it for integration this week, so that it goes in to Moodle 2.6.

Note that this is a major change to how CBM questions work, but I hope that is an major postive change. (Thanks mainly to Tony for pushing me to do this.)

The real difference is that now when you add a CBM question to a quiz with max mark 1, then the grade for a low certainty response will be between 0 and 1, and the grade for a high certainty response will be between -6 and 3.

If you have exiting attempts at a CBM quiz, then they will not change until you regrade them. However, once you regrade them, the marks will change. Simiarly all quiz attempts done after you upgrade to 2.6 will behave in the new way. You have been warned. Assuming this gets into Moodle 2.6, we must remember to add a bit warning about that to the release notes.

 
Average of ratings: -
argm
Certainty-Based Marking (CBM): 2.6 update
Group Particularly helpful Moodlers

Moodle 2.6 now includes most of the improvements in core code that have previously required patches. Please see the relevant Moodle Documentation: http://docs.moodle.org/26/en/Using_certainty-based_marking . Patches are still available for earlier CBM versions (1.9-2.5: www.tmedwin.net/cbm/moodle ) where I will try to maintain uptodate revisions if core changes are made in the relevant files.

Note that in 2.6, it is important to install the approved Quiz Report Plugin for CBM grade reports to teachers. Apparently it is Moodle policy to keep such (readily adaptable) plugins separate from core code.

I have listed the main forum discussion sites concerning CBM in Moodle at www.tmedwin.net/forum .  Things that might still be improved for added efficiency in core code are perhaps :

  • Single-click buttons for both Certainty + Check when using immediate feedback behaviour
  • 'No Idea' button so students can get feedback without being forced to guess
  • Restriction of feedback display in review (for self-tests), to only those Qs attempted

Note that students are now provided with conventional (accuracy) as well as CBM scores, and these scores are displayed both relative to all questions in a quiz and relative to a selection that the student (for self-test purposes) has chosen to use.

 

 

 

 

 
Average of ratings:Useful (1)
argm
Re: Certainty-Based Marking (CBM): updates for 2.4-2.6
Group Particularly helpful Moodlers

There are some changes in the core files requiring patches for CBM in Moodle 2.4.8 and 2.5.4. Appropriate downloadable patches are now available at  www.tmedwin.net/cbm/moodle . Users of 2.6 require a CBM plugin (see above post) but not patches. I have written a summary of the CBM changes made in 2.6 at  www.tmedwin.net/forum .

ps I'm staying near Brisbane till the end of January, and giving a talk on Wed 22nd Jan at UQ St Lucia. If interested, see details at www.tmedwin.net/forum .

 

 

 
Average of ratings: -
argm
Certainty-Based Marking (CBM): updates for 2.5.5
Group Particularly helpful Moodlers

For those upgrading to 2.5.5 the CBM patches incorporating 2 small changes in core code are now available at http://tmedwin.net/cbm

 

 
Average of ratings: -
argm
Certainty-Based Marking (CBM): patches for 2.5.6 etc.
Group Particularly helpful Moodlers

For those upgrading to 2.5.6, new CBM patches are available at http://tmedwin.net/cbm . Other recent Moodle upgrades are compatible with the existing patches. NB Moodle 2.6 & 2.7 don't need patches but require the quiz_cbmgrades plugin 

 
Average of ratings: -
Picture of Joshua Bragg
Re: Certainty-Based Marking (CBM): patches for 2.5.6 etc.
Group Particularly helpful MoodlersGroup Testers
Is there some reason why this cbm quiz report is not included in the the standard Moodle distribution?  It seems like a no brainer to me...

I've just started using CBM and the report really is essential.
 
Average of ratings:Useful (2)
argm
Re: CBM plugin should be in standard Moodle
Group Particularly helpful Moodlers

I agree that the report plugin is essential if people are to use CBM in Moodle 2.6+ (and the patches are essential in earlier versions). The plugin has been downloaded some 360 times now without anyone raising any issues, so the case for incorporation is strong - potentially saving a lot of people both confusion and the bother of installing the plugin. With that in mind, now is the time to say if there should be any improvements, and then I hope this could go ahead.

 
Average of ratings:Useful (2)
argm
CBM Grades Summary plugin for 2.7
Group Particularly helpful Moodlers

A couple of minor edits have proved necessary for the CBM Grades plugin to handle changes in Moodle 2.7. This plugin is essential for teachers using CBM in Moodle 2.6 or 2.7 and is available at https://moodle.org/plugins/view.php?plugin=quiz_cbmgrades . You need  version 1.4 for Moodle 2.7 and version 1.2 for Moodle2.3 - 2.6. If the plugin were in core Moodle it would have been much easier to identify and make these trivial edits, which were required in every Report plugin.

If you use CBM in earlier Moodle versions (1.9-2.5) you must install patches that provide the full functionality, available at http://TMedwin.net/cbm .


 
Average of ratings: -
argm
CBM Grades Plugin and code improvements
Group Particularly helpful Moodlers

I have revised my overview of CBM in Moodle and my own software and the main Moodle CBM documentation , together with the CBM Grades Plugin and downloads available as code improvements:

The CBM Grades Plugin is essential for anyone using CBM. I have updated it for 2.3 - 2.8dev on the Moodle site. Improvements deal with the following issues:
     The plugin used to freeze if you tried to sort on certain columns.
    Clarifies and improves the range of columns, including the CB Grade (based on Accuracy + CB Bonus)
    Presents the data correctly if CBM has been switched on or off since a quiz attempt.
If you are using Moodle 2.3-2.5, the required plugin is now slightly different from the plugin for 2.6. Upgrading should all be initiated automatically by the Moodle plugin site. Apologies that changes have occurred in 3 steps recently.

There are now patches available to improve CBM code in Moodle 2.6 - 2.8dev. See my download site.  
They don't affect function when CBM is not used. Issues dealt with:
    More Help is provided about behaviours when selecting Quiz behaviours ( https://tracker.moodle.org/browse/MDL-46747 )
    Now shows the weighting along with marks for questions (Important if this is changed from default - though I don't recommend that with CBM).
   Presents the data in attempt reviews correctly if CBM has been switched on or off since a quiz attempt.
   Includes the CB Grade (based on Accuracy + CB Bonus) in attempt reviews.
These are all fairly simple code changes, but quite significant improvements. They should really be in core code.

Issue not dealt with:
  I think it would be better if the 'Grade' displayed in the gradebook was the CB grade (based on Accuracy + Bonus) - see this forum discussion .
This is important, but more complicated. I would need help if it were to be done through GIT, from someone used to that. Some of the main code needed (to calculate attempt->sumgrades differently for CBM) is available here , but this needs to be extended to the code that regrades attempts also.
      
 

 
Average of ratings: -
argm
Re: CBM code upgrades: 2.6-2.8
Group Particularly helpful Moodlers

For those upgrading to Moodle 2.8, 2.7.3 or 2.6.6,  code patches for optimal CBM functioning are now available at www.tmedwin.net/cbm/moodle. These improve presentation of attempt reviews with CBM compared with core code, and deal correctly with the situation if CBM behaviour was switched on or off since a reviewed attempt was made. Please see also Essential ADMIN actions for CBM in Moodle Docs for the required staff report plugin and Grade settings.


 
Average of ratings:Useful (1)
argm
Re: CBM code upgrades: 2.8.1
Group Particularly helpful Moodlers

Code patches for the shortlived 2.8.0 have been replaced with those for 2.8.1 at www.tmedwin.net/cbm/moodle (differing only in version documentation). They improve presentation of attempt reviews with CBM, compared with core code, and deal correctly with the situation if CBM behaviour was switched on or off since a reviewed attempt was made. Please see also Essential ADMIN actions for CBM in Moodle Docs for the required staff report plugin and Grade settings.

 
Average of ratings: -