Override rescaling of maximum grade on randomly chosen quiz questions

Override rescaling of maximum grade on randomly chosen quiz questions

by Stewart Schultz -
Number of replies: 12

I am creating quizzes by adding randomly chosen questions from the question bank.  Each question is assigned a maximum grade depending on the difficulty and amount of information asked for.  Questions vary in the maximum grade value.  However, in the resulting quiz, all random questions are automatically given the same grade value of 1.  The maximum (default) grade that I manually assigned them appears to be ignored. 

Can this behavior be overridden, so that each randomly chosen question receives the maximum grade that I assigned it?

The alternative seems like too much work: duplicating the entire category tree under a separate category for 1 point questions, 2 point questions, 3 point questions, ...., and so on.

This behavior appears to be because "The grade for the randomly chosen question will be rescaled so that the maximum grade is what you have chosen as the grade for the Random Question."  However, I can't chose the correct grade for the random question unless I know which random question is ultimately chosen.   So the only workaround would seem to be to recreate a whole new category tree for each maximum grade value present in the question bank, then use those categories to choose random questions from.  This would be a lot of work, and would create a very large and unwieldy category tree.

Average of ratings: -
In reply to Stewart Schultz

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

The quiz reports and the statistics all work on the assumption that each slot in the quiz is worth the same total mark. It would be really hard to change that.

Can you make different categories for easy, medium and hard questions (or something like that) so that the questions in each cateogry have the same maxmark?

Average of ratings: -
In reply to Tim Hunt

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Stewart Schultz -

Not sure I understand "each slot in the quiz". Surely this doesn't mean each question in the quiz has the same total mark?  In my case each "answer slot" does indeed have the same mark.  But, some questions have more "answer slots" than others, and I want the total mark for each question equal to the total number of answer slots in the question.  This is easy if the questions are manually chosen.  But when they are randomly chosen, every question is automatically assigned a maximum value of 1, and there appears to be no way to assign the correct marks to questions because it is impossible to know which question will be randomly chosen.

Average of ratings: -
In reply to Stewart Schultz

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I don't mean "each question in the quiz has the same total mark?"

A 'slot' in the quiz has a very precise meaning if you know the Moodle (2.1) code. Roughtly speaking a slot is a column in the quiz overview report. The concept of 'slot' is useful because it does not matter if the teacher has chosen to 'shuffle questions'.

Anway, there is no way to do what you want.

Average of ratings: -
In reply to Tim Hunt

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Stewart Schultz -

thanks, Tim.

I think I understand -- if a 'slot' is equivalent to a column in the overview report, then it is definitely not the case that 'each slot in the quiz is worth the same total mark' -- clearly each column can have a different total mark, but each question within a column has the same total mark.    So when a test is retaken and questions shuffled, each question still retains its original slot identity.  The slot order remains the same in the overview report, even though the question order changes each time (under the shuffle option).  

It seems to me it would be possible to have questions of different total mark together within the same slot; one would just need to change the way the average score is calculated within that slot.

But if this is not possible or too difficult to implement, then would it be possible to create an option within the quiz edit window, under question bank, to display questions according to total marks?  So e.g. if I choose a quiz category to display, then ask that only questions worth 2 marks are displayed within that category, then I can randomly choose from those questions, and manually assign 2 marks to the resulting questions in the "grade" column of the "questions in this quiz" display.

This would save a lot of work duplicating the category tree for every value of total marks within the question bank.

Average of ratings: -
In reply to Stewart Schultz

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

"clearly each column can have a different total mark, but each question within a column has the same total mark." - This is correct, and what I was trying to say.

"It seems to me it would be possible ... one would just need to ... " - Are you a developer who has read the Moodle code, and hence could you implement this yourself? If not (and there is no reason you should be) then I am afraid you are not qualified ot express an opinion.

If you want this in Moodle right now, you will have to re-orgainse your question bank so all the quetsions with the same total mark are in the same cateogry, then build your quiz from those categories.

Average of ratings: -
In reply to Tim Hunt

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Stewart Schultz -

Rather than have a discussion about the relative qualifications of myself or Tim, I invite folks to comment here.  I would be very interested in opinions from both users and developers on the utility and practicability of either an option to display or sort quiz questions based on total mark value, or to allow questions varying in total mark value within a single quiz slot.

Average of ratings: -
In reply to Stewart Schultz

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Jean-Michel Védrine -

Then if you invite us ...

I find what you suggest quite surprising and quite unfair between students. I don't know in your country, but I am sure french students would protest if I ever wanted to do such a thing!

Why is it unfair ?

Imagine that 2 students  let's call them student A and student B, have a single question in common in their attempts and this question is graded on 3 points.

But the other questions in both attempts are certainly not the sames, and with your system, chance are great that the total grade of student A and student B are not the same : let x be the total grade of student A and y be the the total grade of student B and assume x and y are not equal.

So thecommon question account for 3/x of student's A attempt grade and for 3/y of student's B grade.

But this is the same question !

So they don't get the same "reward" for the same work !!! Quite unfair !!!

So my -1 to get this option in Moodle quiz from a teacher point of view.

Now as a programmer you don't imagine how complicated would be this change, for instance how would overview and statistics reports works ? You certainly have not envisionned all the consequences.

But after all, you are perfectly allowed to have a different opinion than mine big grin.

Then you must understand how open software development works : you must find people interested in this change and either able to do it or to pay somebody else to do it. But even when it is done, it will be a "local Moodle customization" unless the Moodle community find it interesting enought to be included in a future release.

This is how a lot of changes were done in Moodle.

Average of ratings: -
In reply to Jean-Michel Védrine

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Stewart Schultz -

Thanks for the comment, and please allow me respond.

1.  I agree that what you describe is an instance of unfairness.  However, such unfairness is already possible within the randomly chosen questions option, and what I am proposing would reduce it.   At present, if you randomly choose questions from within a pool of questions that differ in total mark, then some students will receive harder questions than others.   This is problematic, right?  We can partially alleviate this problem by allowing those harder questions to receive higher marks.  Then those students with the harder questions at least would get higher marks than those students that did not receive the harder questions.  At present, the quiz module does not allow this.  The purpose of my post was to put this idea to the community of both users and developers, to see if this is desireable to users, and if/how this is possible for developers.

2. If it is not possible to institute the above change, i.e. allow the original total mark value to be retained for each randomly chosen question, then we could solve the problem by allowing the question bank page to display questions according to their total mark (which I pointed out twice in my posts above).   Then, one could choose an option to display, say, only 5 point questions within any subject category, than add random questions from that subset to the exam.  This would assure that each exam slot receives random questions of near-equal difficulty. This option would completely solve both the problem that I point out in (1) above, as well as the problem you point out in your post, wouldn't it?   In other words, if such a display were possible, then both the following are true: all students receive exams of the same difficulty, and 3/x will always equal 3/y.  So, my proposed solution would seem like an improvement that anybody using the randomly chosen question option would find useful, if their goal is to have fair tests for the students.

3.  I understand how open software development works.

Average of ratings: -
In reply to Stewart Schultz

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Jean-Michel Védrine -

At present, if you randomly choose questions from within a pool of questions that differ in total mark, then some students will receive harder questions than others

Why thousands of Moodle quiz users don't have this problem ? Because when we want to use random questions we organise our question bank in a sensible way : (sub)categories with questions of the same difficulty level and having the same maximum grade. Just simple and effective.

If it is not possible to ... then we could solve the problem by ...

How do you know that your 2nd solution need less work than the first one ? Or that it is even possible to do so without huge changes in the code ? Do you know how random questions works ?

How many programmers work hours do you think this would require ?

But may be the best response is to wait for other comments.

I may be proven wrong, but I think you will realise soon that not a lot of Moodle users are interested in this change ...

Average of ratings: Useful (1)
In reply to Jean-Michel Védrine

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Stewart Schultz -

Why thousands of Moodle quiz users don't have this problem ? Because when we want to use random questions we organise our question bank in a sensible way : (sub)categories with questions of the same difficulty level and having the same maximum grade. Just simple and effective.

If there are indeed thousands of Moodle users who are creating extra subcategories for every level of question difficulty, within every subject category, in order to use the randomly chosen question option, then perhaps there are thousands of Moodle users who would be happy not to have to do that anymore?

Average of ratings: -
In reply to Stewart Schultz

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Lev Abramov -

Stewart, there are other, much more important functions of correctly organized categories and subcategories - for example, the ability to share their contents among course creators working on the same course but creating different chapters/units/modules within the course.

Correct category organization also guarantees correct activity backup/restore. I have seen several cases of quizzes losing questions only because course creators used the default categories, instead of creating a well-planned category hierarchy.

Hence the ability to use random questions (something I have never done, to be honest) seems like a by-product rather than the main objective of proper category structure planning and implementation.

Having said that, I must confess that the whole idea of wanting a better/easier/faster/more convenient way of doing things in Moodle is very attractive. With each new version, we seem to get a set of features that not only allows us to do more but also to do it in a less labor-intensive way. Unfortunately, it seems like most of the efforts are invested in extending the list of front-end features, and very few developers really devote enough thought to the amount of work course creators have to invest in their work. Maybe it's time to explicitly attract the attention of the developer community to the fact that teachers (who are usually course creators) are also human beings who deserve a better life, eh?

Moreover: maybe it's time to establish a group that will act as a watchdog ensuring that no feature, no matter how important or effective in terms of online pedagogy, requres more work on the part of the course creators than is really needed.

What say you?

Average of ratings: -
In reply to Tim Hunt

Re: Override rescaling of maximum grade on randomly chosen quiz questions

by Andy Anderson -
I understand the issues here. BUT I would like to see is some intelligence in choosing the grade when creating a random question. If all of the possible questions in the question bank are 4 points, then the random question should be given 4 points! I planned ahead and grouped my questions in categories all with the same points, and it was a lot of unnecessary work for me to have to go through 30 out of 40 questions and reassign their values (Moodle 2.2.1). #WorstCritiquePossible: I didn't have to do this with Blackboard! :-S -- Andy
Average of ratings: -