Marks, accuracy and CBM

Marks, accuracy and CBM

by David Korfhage -
Number of replies: 26

I'm experimenting with Moodle quizzes as formative assessment, and trying CBM.  Based on a blog post I read (and also student reactions), I'm interested in having students see their accuracy (% correct) as well as their CBM adjusted score.  I think they would find that less troubling as well as more useful (e.g., a big gap between their accuracy and their adjusted score would tell them something.  

I've read that there have been proposals for implement this.  Has this been fully developed yet?  Is there a way to do this?

Thanks.

Average of ratings: -
In reply to David Korfhage

Re: Marks, accuracy and CBM

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

You are right. That functionality should be there. Tony Gardiner-Medwin has some code that does it, which you can get from http://www.tmedwin.net/cbm/moodle/. That is not suitable for including in the standard Moodle release, and I have not finished reworkign it. See MDL-32188 for details of that.

Average of ratings: -
In reply to Tim Hunt

Re: Marks, accuracy and CBM

by David Korfhage -

Thanks.  Just to clarify--that functionality will probably be there at some point, but it's not currently something I can do.  Do I understand that correctly?

And are there any workarounds?

 

Average of ratings: -
In reply to David Korfhage

Re: Marks, accuracy and CBM

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Correct, it will happen some time.

The only work-around right now is to install Tony's code, which involved patching core Moodle.

Average of ratings: -
In reply to Tim Hunt

Re: Marks, accuracy and CBM

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Quite a lot of people have downloaded these patches for 1.9, 2.0, 2.2, 2.3, 2.4 from http://www.tmedwin.net/cbm/moodle/ and I haven't had reports of any problems using them. They are pretty innocuous because they don't alter the database, and are easily removed if preferred for any reason. I find it hard to work with current Moodle coding conventions, so would welcome anyone interested in working with Tim and myself to get these patches into core Moodle. Meanwhile the CBM facility at www.ucl.ac.uk/lapt is a good deal more sophisticated in many ways and is open to other institutions to use for student self-tests.   

Average of ratings: Useful (1)
In reply to Tony Gardner-Medwin

Re: Marks, accuracy and CBM

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

I've put up new CBM files for use with the new Moodle versions 2.2.8 and 2.3.5. There were only small changes in relevant core files since 2.2.6, 2.3.3 and none in 2.4.2 compared to 2.4.1. Use the 2.4.1 download if you have 2.4.2. I haven't changed the functionality of the mods. See http://tmedwin.net/cbm/moodle/ .

Average of ratings: -
In reply to Tony Gardner-Medwin

CBM Files for 2.5

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Files with improved CBM functionality for Moodle 2.4, 2.5 (rcl) are now available at my download site  http://tmedwin.net/cbm/moodle/ . There are no recent changes in the functionality of these mods, just updates to incorporate small changes made elsewhere in those core files that need to be replaced. Please let me know (ucgbarg at ucl.ac.uk) if there are any issues or queries. I don't automatically receive emails from this forum. TGM

Average of ratings: -
In reply to Tim Hunt

CBM files for 2.5.1, 2.4.5, 2.3.8, 2.2.11, 1.19.19

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

New patch files for improved CBM functionality in today's Moodle releases are now available at my download/demo site  http://tmedwin.net/cbm/moodle/ . There are a few changes merging minor differences in the core files that you need to replace. No adjustments were needed for the newest versions of Moodle 2.2 or 1.9, so the patches remain the same. 

I recently circulated those who have downloaded these patches in the last couple of years, asking if anyone has had problems. None were reported. If you know of any problems, even minor ones, please report them to me, ucgbarg at ucl.ac.uk, and if you like, raise them here. Some of the enhancements of functionality are substantial and without them CBM in Moodle may unfortunately seem rather obscure.

Average of ratings: -
In reply to Tony Gardner-Medwin

Re: CBM files for Moodle 2.5.2, 2.4.6, 2.3.9

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

In Moodle 2.5.2 and 2.4.6 (just released) there are minor changes in 3 of the core files that need to be replaced for proper CBM functionality. The required patch files for these versions are  now available at  http://tmedwin.net/cbm/moodle/ . The patches for 2.3.8 are still valid for 2.3.9 since there are no changes in the relevant files. Let me know of any problems using these patches. TGM

 

Average of ratings: -
In reply to Tony Gardner-Medwin

Re: CBM files for Moodle 2.5.2, 2.4.6, 2.3.9

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

OK, so I finally made some time to work on MDL-32188, in the hope of getting it done for Moodle 2.6 (which means that it needs to be done this week.)

I just made a long comment there. If you are interested in this, any help you can give witht the outstanding issues would be apprectiated.

Average of ratings: -
In reply to Tim Hunt

Re: CBM files for Moodle 2.5.2, 2.4.6, 2.3.9

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I am making progress. I think the code on MDL-32188 is basically done, and I plan to test it a bit more tomrrow, and then submit it for integration this week, so that it goes in to Moodle 2.6.

Note that this is a major change to how CBM questions work, but I hope that is an major postive change. (Thanks mainly to Tony for pushing me to do this.)

The real difference is that now when you add a CBM question to a quiz with max mark 1, then the grade for a low certainty response will be between 0 and 1, and the grade for a high certainty response will be between -6 and 3.

If you have exiting attempts at a CBM quiz, then they will not change until you regrade them. However, once you regrade them, the marks will change. Simiarly all quiz attempts done after you upgrade to 2.6 will behave in the new way. You have been warned. Assuming this gets into Moodle 2.6, we must remember to add a bit warning about that to the release notes.

Average of ratings: -
In reply to David Korfhage

Certainty-Based Marking (CBM): 2.6 update

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Moodle 2.6 now includes most of the improvements in core code that have previously required patches. Please see the relevant Moodle Documentation: http://docs.moodle.org/26/en/Using_certainty-based_marking . Patches are still available for earlier CBM versions (1.9-2.5: www.tmedwin.net/cbm/moodle ) where I will try to maintain uptodate revisions if core changes are made in the relevant files.

Note that in 2.6, it is important to install the approved Quiz Report Plugin for CBM grade reports to teachers. Apparently it is Moodle policy to keep such (readily adaptable) plugins separate from core code.

I have listed the main forum discussion sites concerning CBM in Moodle at www.tmedwin.net/forum .  Things that might still be improved for added efficiency in core code are perhaps :

  • Single-click buttons for both Certainty + Check when using immediate feedback behaviour
  • 'No Idea' button so students can get feedback without being forced to guess
  • Restriction of feedback display in review (for self-tests), to only those Qs attempted

Note that students are now provided with conventional (accuracy) as well as CBM scores, and these scores are displayed both relative to all questions in a quiz and relative to a selection that the student (for self-test purposes) has chosen to use.

 

 

 

 

Average of ratings: Useful (1)
In reply to Tony Gardner-Medwin

Re: Certainty-Based Marking (CBM): updates for 2.4-2.6

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

There are some changes in the core files requiring patches for CBM in Moodle 2.4.8 and 2.5.4. Appropriate downloadable patches are now available at  www.tmedwin.net/cbm/moodle . Users of 2.6 require a CBM plugin (see above post) but not patches. I have written a summary of the CBM changes made in 2.6 at  www.tmedwin.net/forum .

ps I'm staying near Brisbane till the end of January, and giving a talk on Wed 22nd Jan at UQ St Lucia. If interested, see details at www.tmedwin.net/forum .

 

 

Average of ratings: -
In reply to Tony Gardner-Medwin

Certainty-Based Marking (CBM): updates for 2.5.5

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

For those upgrading to 2.5.5 the CBM patches incorporating 2 small changes in core code are now available at http://tmedwin.net/cbm

 

Average of ratings: -
In reply to Tony Gardner-Medwin

Certainty-Based Marking (CBM): patches for 2.5.6 etc.

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

For those upgrading to 2.5.6, new CBM patches are available at http://tmedwin.net/cbm . Other recent Moodle upgrades are compatible with the existing patches. NB Moodle 2.6 & 2.7 don't need patches but require the quiz_cbmgrades plugin 

Average of ratings: -
In reply to Tony Gardner-Medwin

Re: Certainty-Based Marking (CBM): patches for 2.5.6 etc.

by Joshua Bragg -
Is there some reason why this cbm quiz report is not included in the the standard Moodle distribution?  It seems like a no brainer to me...

I've just started using CBM and the report really is essential.
Average of ratings: Useful (2)
In reply to Joshua Bragg

Re: CBM plugin should be in standard Moodle

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

I agree that the report plugin is essential if people are to use CBM in Moodle 2.6+ (and the patches are essential in earlier versions). The plugin has been downloaded some 360 times now without anyone raising any issues, so the case for incorporation is strong - potentially saving a lot of people both confusion and the bother of installing the plugin. With that in mind, now is the time to say if there should be any improvements, and then I hope this could go ahead.

Average of ratings: Useful (2)
In reply to Tony Gardner-Medwin

CBM Grades Summary plugin for 2.7

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

A couple of minor edits have proved necessary for the CBM Grades plugin to handle changes in Moodle 2.7. This plugin is essential for teachers using CBM in Moodle 2.6 or 2.7 and is available at https://moodle.org/plugins/view.php?plugin=quiz_cbmgrades . You need  version 1.4 for Moodle 2.7 and version 1.2 for Moodle2.3 - 2.6. If the plugin were in core Moodle it would have been much easier to identify and make these trivial edits, which were required in every Report plugin.

If you use CBM in earlier Moodle versions (1.9-2.5) you must install patches that provide the full functionality, available at http://TMedwin.net/cbm .


Average of ratings: -
In reply to Tony Gardner-Medwin

CBM Grades Plugin and code improvements

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

I have revised my overview of CBM in Moodle and my own software and the main Moodle CBM documentation , together with the CBM Grades Plugin and downloads available as code improvements:

The CBM Grades Plugin is essential for anyone using CBM. I have updated it for 2.3 - 2.8dev on the Moodle site. Improvements deal with the following issues:
     The plugin used to freeze if you tried to sort on certain columns.
    Clarifies and improves the range of columns, including the CB Grade (based on Accuracy + CB Bonus)
    Presents the data correctly if CBM has been switched on or off since a quiz attempt.
If you are using Moodle 2.3-2.5, the required plugin is now slightly different from the plugin for 2.6. Upgrading should all be initiated automatically by the Moodle plugin site. Apologies that changes have occurred in 3 steps recently.

There are now patches available to improve CBM code in Moodle 2.6 - 2.8dev. See my download site.  
They don't affect function when CBM is not used. Issues dealt with:
    More Help is provided about behaviours when selecting Quiz behaviours ( https://tracker.moodle.org/browse/MDL-46747 )
    Now shows the weighting along with marks for questions (Important if this is changed from default - though I don't recommend that with CBM).
   Presents the data in attempt reviews correctly if CBM has been switched on or off since a quiz attempt.
   Includes the CB Grade (based on Accuracy + CB Bonus) in attempt reviews.
These are all fairly simple code changes, but quite significant improvements. They should really be in core code.

Issue not dealt with:
  I think it would be better if the 'Grade' displayed in the gradebook was the CB grade (based on Accuracy + Bonus) - see this forum discussion .
This is important, but more complicated. I would need help if it were to be done through GIT, from someone used to that. Some of the main code needed (to calculate attempt->sumgrades differently for CBM) is available here , but this needs to be extended to the code that regrades attempts also.
      
 

Average of ratings: -
In reply to Tony Gardner-Medwin

Re: CBM code upgrades: 2.6-2.8

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

For those upgrading to Moodle 2.8, 2.7.3 or 2.6.6,  code patches for optimal CBM functioning are now available at www.tmedwin.net/cbm/moodle. These improve presentation of attempt reviews with CBM compared with core code, and deal correctly with the situation if CBM behaviour was switched on or off since a reviewed attempt was made. Please see also Essential ADMIN actions for CBM in Moodle Docs for the required staff report plugin and Grade settings.


Average of ratings: Useful (1)
In reply to Tony Gardner-Medwin

Re: CBM code upgrades: 2.8.1

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Code patches for the shortlived 2.8.0 have been replaced with those for 2.8.1 at www.tmedwin.net/cbm/moodle (differing only in version documentation). They improve presentation of attempt reviews with CBM, compared with core code, and deal correctly with the situation if CBM behaviour was switched on or off since a reviewed attempt was made. Please see also Essential ADMIN actions for CBM in Moodle Docs for the required staff report plugin and Grade settings.

Average of ratings: -
In reply to Tony Gardner-Medwin

Re: CBM code upgrades for 2.7-2.9

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Recent moodle versions 2.7.9, 2.8.7, 2.9.1 have been checked against existing CBM code patches, which don't require alteration to work with these new versions. Patches are available at www.tmedwin.net/cbm/moodle . These improve presentation compared with core code when reviewing attempts with CBM, and (unlike core code) they deal correctly with the situation if CBM behaviour was switched on or off since the reviewed attempt was made. Please see also Essential ADMIN actions for CBM in Moodle Docs for plugin requirements to ensure proper staff reports and for required Grade settings when using CBM.

Average of ratings: -
In reply to Tony Gardner-Medwin

CBM code upgrades for 2.7-2.9

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

The patches that make CBM work properly in Moodle ( www.tmedwin.net/cbm/moodle ) have been checked with Moodle versions 2.7.11, 2.8.9, 2.9.3 launched today. No new changes were necessary for these core updates, but CBM users are reminded to include the patches (see also Essential ADMIN actions for CBM in Moodle Docs). Many users are confused by the fact that CBM Moodle grades are by default calculated just from total CBM marks and go up to 3 times the normal 'maximum' grades. To make CBM grades correspond to CBM Accuracy (up to 100%, and a more intuitive measure of performance) use the 'plus' version of these patches.

Average of ratings: -
In reply to Tony Gardner-Medwin

CBM code upgrades for 2.7-3.0.2

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

The patches that make CBM work properly in Moodle ( www.tmedwin.net/cbm/moodle ) have been checked with the latest Moodle versions (3.0.2, 2.9.4,2.8.10,2.7.12). Though no new changes were required, if you upgrade you should make sure you use the correct patches. Let me know of any problems (ucgbarg@ucl.ac.uk).


Average of ratings: -
In reply to Tony Gardner-Medwin

Re: CBM code upgrades for 2.9.5 and 3.0.3

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

The patches that make Certainty-Based Marking (CBM) work properly in Moodle ( www.tmedwin.net/cbm/moodle ) have been checked with the latest Moodle versions. Minor changes were required for versions 3.0.3, 2.9.5 (simply because of a typo that was corrected in core code). Patches for 2.8.11, 2.7.13 are unchanged from earlier versions. If you upgrade to any of these core versions you should make sure you use the correct patches from www.tmedwin.net/cbm/moodle. Let me know of any problems (<u>ucgbarg@ucl.ac.uk</u>).


Average of ratings: Useful (1)
In reply to Tony Gardner-Medwin

CBM code upgrades for 3.1 etc.

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Moodle 3.1 has a small change in core code that requires a new version of the files to make Certainty-Based Marking (CBM) work best in Moodle.  Downloads are available at  www.tmedwin.net/cbm/moodle . The other recent upgrades (to 3.0.4, 2.9.6, 2.8.12, 2.7.14) allow the same CBM patch files to be used as their immediate predecessors. Other links to important information for users of CBM (essential PLUGIN, etc.) are at www.tmedwin.net/cbm .  Let me know of any problems ( ucgbarg@ucl.ac.uk ) since I don't routinely view this forum.

Average of ratings: -