Can I sample answers from a Multiple Choice Question?

Can I sample answers from a Multiple Choice Question?

by Oleg Melnikov -
Number of replies: 31

Hi All. Is there a way to have a (multi-answer) Multiple Choice question with 4 answer choices pooled from a larger pool of associated answers. 

For example, I might have a question "What are the ocean colors?" with the possible pool of answers "blue", "green", "turquois", "orange", "purple", "rainbow", "apple", ...

But I would like only 4 of these choices to be presented to the student (so as to further individualize/automate the quiz). I think this might be possible with Formulas type of question, but is it possible with base Moodle (without any extra plug-ins) ? For example, might this be possible with Multiple Choice question or Calculated Question ?

Average of ratings: -
In reply to Oleg Melnikov

Re: Can I sample answers from a Multiple Choice Question?

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
You are not the first to ask for this kind of feature. However, I am not sure if there is a Tracker feature request for it. I recall a discussion about allowing "text" (or words) to be used within some of the "calculated" question types that use variables.
In reply to Oleg Melnikov

Re: Can I sample answers from a Multiple Choice Question?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
As Rick says, people sometimes ask for it, but no-one has ever built it. (And it would be quite easy to do this as a new question-type plugin .)
In reply to Tim Hunt

Re: Can I sample answers from a Multiple Choice Question?

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
Use random questions?
In reply to Dominique Bauer

Re: Can I sample answers from a Multiple Choice Question?

by Oleg Melnikov -
Yes, question randomization is one option, which I do already use. Still often I encounter questions, which could be feeding from a single pool of answers, thereby dramatically increasing the variety of questions. For example, I could have 2 identical questions with choices A,B,C,D and E,F,G,H. Or I could have a single question with 4 choices drawn from the answers A,B,C,D,E,F,G,H. The latter produces many more permutations or 8C4=70 combinations (if we ignore the order) of answer choices. That's 70x lift in count of (somewhat) unique questions from a simple merge of 2 multi-answer questions.

One example of such question could be:
"Which of the following 4 scenarios is an example of unsupervised machine learning?" The 4 choices are draw from
1. scenario 1
...
N. scenario N
In reply to Oleg Melnikov

Re: Can I sample answers from a Multiple Choice Question?

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

And it would be quite easy to do this as a new question-type plugin ... for someone who knows what he's doing, or maybe for someone who only has a background in programming but could count on a little help if needed.

Here is how I see it. The new question-type plugin could be created from the multiple choice question.

If the correct choices have different values, for example 25% and 50% so that 25% x two choices + 50% x one choice = 100%, you have to consider different question banks for the different values. To keep things simple, at least as a first step, let's assume that the correct choices all have the same value.

Let's introduce the following variables:

  • n:      the number of correct choices presented to the student
  • np:    the number of correct choices in the choice bank
  • m:     the number of incorrect choices presented to the student
  • mp:   the number of incorrect choices in the choice bank

The following restrictions apply:

  • n >= 1
  • m >= 1
  • np >= n
  • mp >= m

The total number of choices presented to the student is equal to n + m.

In the question settings, the teacher sets the following options:

  • Total number of choices presented to the student, n + m
  • Number of correct choices presented to the student, n

More or less the same algorithm applies when there is only one correct choice or several correct choices.

The positive values of the correct choices are limited to the single value 1/n, for example 50 % when n = 2. Different negative values can be chosen by the teacher for the incorrect choices.

The np correct choices are identified by their positive value. Of these choices, n choices are chosen at random.

The mp incorrect choices are identified by their null or negative values. From these choices, m choices are chosen at random.

The n and m choices are presented to the student, in order or shuffled.

The student's result is the total of the points obtained for choices n and m.

Average of ratings: Useful (1)
In reply to Dominique Bauer

Re: Can I sample answers from a Multiple Choice Question?

by Oleg Melnikov -
Dominique, yes, this would be great? Does anything of this sort exist already?
In reply to Oleg Melnikov

Re: Can I sample answers from a Multiple Choice Question?

by Dominique Bauer -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
I do not think so. It's just the way I imagine a new plugin could be made.
In reply to Oleg Melnikov

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Oleg,

I've e-mailed you to the email address in your Moodle profile. Looking forward.

Joseph

In reply to Dominique Bauer

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

@Dominique,

I've started work on the lines you suggest, but based on the All-or-Nothing Multiple Choice question type instead.

You write "To keep things simple, at least as a first step, let's assume that the correct choices all have the same value." Basing development on the All-or-Nothing Multiple Choice question type solves that question.

I will report on my progress here, and ask for opinions about options to be offered on the edit screen.

I have e-mailed Oleg about it, but got no answer so far.

Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Change of plans.wink

Just noticed that the All-or-Nothing Multiple Choice question type is based on the OU multiple response type.

I'll try to base my "random_select" type on the latter instead.

Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Oleg Melnikov -
Joseph, thank you. Good find. I'm looking into this.
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Oleg Melnikov -
Hi Joseph. You are correct, All-or-Nothing MC and OU MR are nearly identical. Either one would be a good base for the Sampled Answer Multiple Choice. Is this a lot of work? I can help evaluate or test it on my local Moodle install. Thanks in advance!!

Btw, I checked for your email, but did can't find it. You can message me via Moodle inbox, if needed. Or try the email again.
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

I am making good progress on a new qtype (based on the the OU multiple response type).

In my question.php script I define a new, reduced, list of answers LIST B (randomly selected from the current question's answers list LIST A).

I need to use list B in the renderer.php script. What is the best policy to pass LIST B from question.php to renderer.php?

So far I've been successfully using global $SESSION. Is that OK? Are there any drawbacks? Is there a better way?

Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Definitely don't use session!

What you should copy is how the renderer gets the random order that is set in start_attempt.
In reply to Tim Hunt

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators
Thanks for your prompt reply, Tim. I thought so!
I do set the "random order" in question.php start_attempt(), but... I have no idea how to "copy" that random order in the renderer. Do you mean I should actually put inside the renderer an exact copy of the script I use in question.php? I have tried that solution but it does not work, because that little script involves shuffling the answers array, and it's shuffled one way in question.php and another way in the renderer, so those 2 random order arrays do not match at runtime!
Rather than using $SESSION (which works fine, by the way), is there a way to use a global variable, which I could set in question.php and retrieve in the renderer? I've tried with no success.
Any suggestions welcome!
Joseph
Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
If you just try to understand oumultiresponse, you will only be seeing a small fraction of the picture, becuase it is built on the core multiplechoice type. You need to understand a bit more about how the core question type works.

I think you should be able to achieve what you want just by overriding start_attempt and apply_attempt_state. If you do that right, then probably you don't need to change the renderer at all.

And, probably, more important that trying to follow what the code is doing, focus on the data structure. What do the various files of the class mean? https://github.com/moodle/moodle/blob/master/question/type/multichoice/question.php#L60(Hmm. Wouldn't it be nice if the person who created this question type had bothered to write PHPdoc comments. I wonder who should be blamed for that wink)
In reply to Tim Hunt

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Thanks for your suggestion, Tim, but I still have not succeeded to achieve what I need except by using the $SESSION global variable.

So for the time being my new question type, which I named "Random select answers", does work at least as a Proof of Concept. It's available on my github at https://github.com/rezeau/moodle-qtype_answersselect. I have not totally "cleaned" it of its OU references.

A sample quiz is available on my Moodle test site at: https://www.rezeau.org/moodle/mod/quiz/view.php?id=75 (login as student01, pasword Student01!) Test course 02.

Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Replying to myself... Tim was right (of course). Using the $SESSION global variable only works when the student is answering the question. It no longer works when reviewing the attempt.

I will definitely have to find another solution... maybe with some help from the Moodle question type gurus.wink

Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Oleg Melnikov -
Joseph, thanks for catching this! Yes, it'd be great, if someone would advise on proper workaround this variable. Can any OU developer help out?
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

I have now hopefully fixed the problem of displaying only the list of the correct answers randomly selected from the answers "pool" at runtime, without resorting to the $SESSION variable.

I have pushed a new version to my github. This needs more testing and some tidying up.

I also have no idea how the tests work, especially behat, etc. so will probably remove the tests directory unless I manage to understand it.

Testers welcome. at https://github.com/rezeau/moodle-qtype_answersselect

Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

New URL for my Quiz to test the Random answers select question type here:

https://www.rezeau.org/moodle/mod/quiz/view.php?id=78

This new Quiz illustrates a new option for displaying correct answers after checking. You can display the correct answers :

    separated by commas (this is the default option)
    separated by spaces (which may allow a whole sentence to be displayed on the same line without extra commas)
    separated by line breaks (useful for long answers)

For potential testers, I have put a new version on my github.

Unfortunately, as I have added the new option mentioned above in the edit form (and thus a new field in the database), to see this option, you will need on your moodle test site to first delete all the questions you had already created, AND uninstall the Random select answers question type in the admin panel BEFORE reinstalling the new version.
Normally, database changes should be automated, but I didn't see fit to do that for an alpha/beta version. I'm getting lazy.wink

 

Average of ratings: Useful (2)
In reply to Oleg Melnikov

Re: Can I sample answers from a Multiple Choice Question?

by Oleg Melnikov -
Big thanks to everyone for ideation and help in shaping this new plugin. 
Kudos to Joseph for writing the code and bringing it to life. 
Now that the plugin is awaiting approval (CONTRIB-8624) to be added to Moodle Plugin Directory (MPD), what can improve it's chances? 
Also, how long does approval process typically take?

Once added to MPD, our LMS admins at Higher School of Economics (HSE) will gladly add it and make it accessible to instructors. Personally, I already have plentiful of uses for this plugin in my Fall semester.

In reply to Oleg Melnikov

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators
Hi Oleg (and others)
I'm pleased to report that my Random answers select question type plugin was approved today and is available from the plugins repository at https://moodle.org/plugins/qtype_answersselect.
I will shortly write the relevant documentation in the Moodle docs wiki.
Joseph
PS.- A lot of thanks to Marcus Green for his help with the testing process.
 
Average of ratings: Useful (6)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Oleg Melnikov -
Hi Joseph. That's great news, indeed! Thanks for all the awesome work! I saw the update in the morning and sent good news and install request to our Moodle admin team. Fingers crossed, it will be installed before semester starts smile I'll keep you posted in case of any issues or questions. Thanks again!
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

I have now completed the documentation for this question type at Random_select_answers_question_type.

A companion plugin to this question type is the 'Random select answers format' plugin which takes a Moodle XML file and imports all regular multichoice questions changing them into 'Random select answers' questions. This enables users to export some of their multichoice questions using the normal Moodle XML export format and import them back with the 'Random select answers' format.

This plugin is currently waiting for approval in the Moodle plugins directory. It's available from my Github at https://github.com/rezeau/moodle-qformat_answersselect

Average of ratings: Useful (5)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Detailed procedure

Aiken is only for multiple-choice questions with a single response. If you want to import more complex things, use GIFT.
Here is how to do things.

  1. Create your MULTICHOICE, MULTIANSWERS questions in GIFT format.
  2. In your Moodle site, go to the questions bank, import and import that GIFT text file as GIFT format (put all those questions in the same category, e.g. "my gift questions"
  3. Your GIFT questions will have been converted into MOODLE MULTICHOICE, MULTIANSWERS questions.
  4. Export your MOODLE MULTICHOICE, MULTIANSWERS questions to the MOODLE XML format
  5. On your Moodle site, install my moodle-qformat_answersselect from my Github at https://github.com/rezeau/moodle-qformat_answersselect
  6. Now import those questions using the Random select answers format.

See the Moodle docs at https://docs.moodle.org/311/en/Random_select_answers_question_type#Import_format

Average of ratings: Useful (2)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Aric Joshua -
That is fantastic news! Thank you for all of your hard work! In the morning, I saw the update and informed our Moodle admin staff of the good news, as well as a request to install it. Fingers crossed, it'll be up and running before the semester begins. I'll keep you updated if there are any problems or queries. Thank you once again!(Edited by Marcus Green - original submission Tuesday, 21 September 2021, 8:22 AM)
Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

In reply to some requests for better import facility from GIFT to the "Random select answers" question type I have developed another question import plugin, qformat_gift_answersselect.

It is available in its alpha, 0.2 release version on my github at https://github.com/rezeau/moodle-qformat_gift_answersselect

Please refer to the official Moodle documentation for GIFT at GIFT_format for more details. Please note that is only imports from GIFT; it does not export.

  1. It will only import from a GIFT file those questions formatted according to the Multiple choice or Multiple choice with multiple right answers.
  2. It will also accept the following simplified format:
    What two people are entombed in Grant's tomb? {
       ~No one
       =Grant
       =Grant's wife
       ~Grant's father
    }
  3. It will accept the various [...format...] marks: Options are [html], [moodle], [plain] and [markdown].
  4. Optionally, it's possible to specify for each question which "Number of correct and incorrect answers" mode to use among: Use all answers (default mode), The Manual selection mode, The Automatic random selection mode by adding [manual] or [auto] just at the beginning of the question text (either before OR after the [html] or [markdown] etc. format markers (if any). If none of those "selection modes" is use, the import will default to the default mode (Use all answers). Please refer to the Random select answers documentation for more, at Random_select_answers_question_type.
  5. When this new import plugin will have been tested by more people I will submit it for inclusion in the Moodle official plugins repo.
Average of ratings: Useful (4)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

I'm pleased to report that my "Random select answers format" plugin has just been approved and is available in the plugins repo at https://moodle.org/plugins/qformat_answersselect

Average of ratings: Useful (3)
In reply to Joseph Rézeau

Re: Can I sample answers from a Multiple Choice Question?

by Oleg Melnikov -
Joseph, thanks for an awesome and exceptionally helpful plugin!!! We have it installed on 2 servers and being used in all 3 of my courses at Higher School of Economics!