Unwanted CBM Marks Showing Up for Moodle Quiz

Unwanted CBM Marks Showing Up for Moodle Quiz

by Christina Dempsey -
Number of replies: 6

I have a professor that I am working with who gave a quiz. She gave students 2 attempts, and also had it on adaptive mode. I'm not sure if those two things are important, but I wanted to include that information. After students took the test, she went to grade it and found grades that were way off. Moodle started applying CBM marks, but she didn't select that option when creating the quiz. Now she needs to get rid of them and wants to know why they appeared. What am I missing here? Did any of the settings that she chose make CBM marks automatically appear? I haven't worked with this before, so any help would be great appreciated. I am attaching two photos of of the student's quiz in question 

Notice here how high the grade is.


This image shows the actual attempt with the CBM markings.


Average of ratings: -
In reply to Christina Dempsey

Re: Unwanted CBM Marks Showing Up for Moodle Quiz

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
For this to have happened, the teacher must have had How questions behave: Deferred feedback with certaintiny based marking at the time when the sutdent started the quiz attempt.

There is no easy way to fix that. I believe it can be fixed if you are prepared to do dangerous messing about with the raw data in the Moodle database. If you want to risk that, I can give more info.

Before releasing your quiz to students, you should always preview it yourself.
Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Unwanted CBM Marks Showing Up for Moodle Quiz

by Christina Dempsey -
Thanks for your reply. When the students took it, the quiz was only in adaptive mode. She didn't change it after the fact. This is why I am so confused about what happened. Is there any other setting that would make those CBM Marks show up besides the question behavior option?
In reply to Christina Dempsey

Re: Unwanted CBM Marks Showing Up for Moodle Quiz

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
The only thing that would affect this is the question behviour quiz setting at the time when the student started the quiz attempt. I have just checked what the code is doing, and I can confirm this.
In reply to Tim Hunt

Re: Unwanted CBM Marks Showing Up for Moodle Quiz

by Christina Dempsey -
Okay, I appreciate your response. When you get a moment, could you please send me more info about looking at the raw data?

Thanks again!
In reply to Christina Dempsey

Re: Unwanted CBM Marks Showing Up for Moodle Quiz

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

OK, so a good query to see almost all the raw data about one quiz attempt is https://docs.moodle.org/dev/Overview_of_the_Moodle_question_engine#Detailed_data_about_an_attempt.

What I am expecting that you will find is that the qu.preferredbehaviour and qa.behaviour columns are all 'deferredcbm'.

If so, it may be possible to fix it as follows. However, this is an educated guess, and I strongly recommend you test it on some data that you don't mind corrupting before doing it for real.

One way to test would be if you can take a copy of your entire Moodle site onto a test server. (This is a good thing to know how to do, becuase this is essentialy what you would have to do if you ever needed to restore your Moodle site from a backup. The steps are a combination of https://docs.moodle.org/38/en/Site_backup and https://docs.moodle.org/38/en/Moodle_migration. You probably already know this, but it does not hurt to repeat it.)

The other way you could could test the steps below is to make a new test course on your real server. Make a quiz in there, set to 'Deferred feedback with CBM behaviour'. Log in as a student and make an attempt. Then you can test with that attempt. However, testing anything on your live server is always risky.

OK, so the steps that might possibly let you change the question behaviour after an attempt has been started:

  1. Find the one row in the question_usages table that corresponds to the quiz attempt of interest. This will be the one where question_usages.id = quiz_attempts.uniqueid.
  2. In this row, change preferred_behaviour to the correct value for the behaviour you want. E.g. 'adaptive'.
  3. The find the corresponding rows in the question_attempts table. There should be one row for each question in the quiz. These are the rows where question_attempts.questionusageid = question_usages.id.
  4. In each of these rows that currently have the wrong behaviour set, change it to the right one.
  5. Then, log in to Moodle. Go to the quiz -> Results -> Grades report, and re-grade the attempt.
  6. Re-run the database query I linked to above, which shows all the raw data, to see what the affect of doing that is.
  7. In Moodle, review the quiz attempt and see how it looks.

As I say, this might work, even thoug it is not supported. Step 4 is tricky, becuase you have to know what the right behaviour is. For example, if you have any Essay questions in your quiz, then will always use 'manualgraded' behaviour and you should never change that. Similarly, if you have any 'Description' items, the behaviour should never be changed from 'informationitem'. Then for some combination of preferredbehaviour and question type, a different behaviour will be used in the question_attempt. For example if the preferredbehaviour is 'interactive' and the question type is something like 'match', then the question_attempt behaviour will be 'interactivecountback'.

Acutally, probably the reliable way to find what the behaviour should be for each question_attempt is to change the quiz setting to the one you want, an then start a new attempt at the quiz. Then you can see which behaviour is used for each question_attempt in that new quiz attempt.

Note, there is a feature request to implement this 'Change behaviour of an attempt that was already made' thing proplery. It is MDL-37949. However, don't hold your breath for that.

Average of ratings: Useful (1)
In reply to Christina Dempsey

Re: Unwanted CBM Marks Showing Up for Moodle Quiz

by Tony Gardner-Medwin -
Picture of Particularly helpful Moodlers Picture of Plugin developers
Has this problem been solved? Please say if it seems indicative of any kind of bug, rather than alterations to settings or data made after the student's attempt. Different parts of the display are strangely inconsistent, even for CBM - most conspicuously 45 responses in the grid with apparently 40 Qs in the quiz.

The grade that the teacher wants from the attempt is presumably the displayed accuracy (=95.0%), which corresponds correctly to the break-down data shown (38/40 Qs correct). The CBM average score according to this breakdown would be 2.55 per Q if all Qs had equal weighting. This would yield a Grade =255.0 relative to all correct at C=100. I'm not sure what might have happened if a student was allowed second attempts at Qs after getting them wrong, using immediate feedback with CBM. Permitting this wouldn't make much sense and I don't think it used to be possible. But I haven't myself looked at (or indeed used) this code in a long time. The "45 vs 40 Qs" issue makes me think that edits may have been made after the attempt took place.