CLOZE multichoice multiple answers

CLOZE multichoice multiple answers

by Davo Smith -
Number of replies: 27
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I've been asked by a customer to extend the Cloze / Embedded / Multianswer question type to add support for 'multiple answer' multichoice questions (in addition to the long-standing support for 'single answer' multichoice questions).

I've added an initial version at: https://tracker.moodle.org/browse/MDL-3782 which acceptes questions in the following format:

Please select the fruits {1:MULTICHOICE_MULTI:=Apple#Good~%-50%Burger~%-50%Hot dog#Not a fruit~%-50%Pizza~=Orange#Correct~=Banana} and vegetables {1:MULTICHOICE_MULTI:=Raddish#Good~%-50%Chocolate~%-50%Biscuit#Not a vegetable~%-50%Cheese~=Carrot#Correct}

I'm looking for some feedback on this.

Thanks.

Average of ratings: Useful (2)
In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Germán Valero -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

It is good news to see the cloze question type upgraded.

I think it would be even better if someone (perhaps Davo?) would write a cloze ditor plugin for Atto, similar to the old cloze editor for TinyMCE, but expanded to include all the new accepted question types for the (new default) Atto editor.

Quite a few Moodlers (Me and 700+) are still using the TinyMCE cloze editor to write their cloze questions.

Having a friendly cloze editor would be a very useful addition for Atto.

In reply to Germán Valero

Re: CLOZE multichoice multiple answers

by Sarra Lev -

I would be very grateful if it were possible to create the following scenario:

Selecting A = 20% (0% awarded if this response is not selected)
Selecting B = 20% (0% awarded if this response is not selected)
Leaving C unselected = 20% (0% awarded if this response is selected)
Selecting D = 20% (0% awarded if this is not selected)
Leaving E unselected = 20% (0% awarded if this is selected)

For example:

Question: Canada is:

Choice #1: North of the US - (20%)

Choice #2: A country - (20%)

Choice #3: Cold all year-round - (20% if they don't choose this)

Choice #4: More populated than the US - (20% if they don't choose this)

Choice #5: A monarchy - (20% if they don't choose this)

Etc. 

Is there an option to do this?

Thanks so much,

SL

In reply to Sarra Lev

Re: CLOZE multichoice multiple answers

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

You would structure it this way:

Choice 1:  50%

Choice 2: 50%

Choice 3: -33%

Choice 4: -33%

Choice 5: -33%

This should have about the same mathematical effect...


In reply to Emma Richardson

Re: CLOZE multichoice multiple answers

by Sarra Lev -

Actually, that does not work the same way at all, except in the scenario that the student got all the answers correct.

If a student marked all of the choices in my scenario they would get 40% (20% for each of the right choices, and 0% for the wrong ones they marked right). In yours they would get 0%

If a student marked one correct answer and three incorrect ones in my scenario, they would get 20%. In yours they would get -50%.



In reply to Sarra Lev

Re: CLOZE multichoice multiple answers

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

You are correct - as you can tell, math is not my strong point!!  

Sorry about that.  I had not worked it all the way through.  That is a hard one to get just right because you cannot assign a grade for a non-selection.  There is an all or nothing question type but that would achieve the same as I put above and not give you the grade you want.  

What if you assigned 50% each to the correct and then -20 to the incorrect - that might get you closer.  If they marked all, they would get 40%.  However if they marked one right and three wrong they would get -10, not 20.  

Not sure if you could set the two correct ones to be over 100% combined (where the question would then just give them 100, I think).  

In reply to Emma Richardson

Re: CLOZE multichoice multiple answers

by Sarra Lev -

Math isn't my strong point either. I had to run it by someone else to make sure I was right. smile

I'm just dying for someone to create this option. I don't think the minus system will work for me. In the meantime I'm using a bunch of terrible work-arounds.

Thanks for trying to help!

In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

My main feedback is surprise that someone finds the question type insufficiently complex smile

In reply to Marcus Green

Re: CLOZE multichoice multiple answers

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

I agree - I use the CLOZE question a lot and I think it is quite robust enough as is and adding all this would make it harder for teachers to grasp.  We already have a multiple answer choice question - does it really need to be in CLOZE too?  

Now, I do whole heartedly agree that an ATTO editor plugin would be nice - even better would be to just build that into the question type itself so that no-one actually had to code them...

Average of ratings: Useful (1)
In reply to Emma Richardson

Re: CLOZE multichoice multiple answers

by Davo Smith -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I'm not quite clear about why adding an extra, optional question subtype (that no one ever has to use if they don't want to), makes the question type more complex to use.

The multiple-answer question subtype is no more difficult to use than any of the other subtypes (it used identical syntax to the single-answer subtypes), it just adds an extra option for those that want to use it.

In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

It will not make it any more difficult to use for anyone who doesn't want to use it, and if you or your employer/client want to develop it, go ahead. My concern is if it were to become part of core Moodle.  If that happens it would add a legacy of maintaining already complex code for a question type that in my view damages the reputation of Moodle as something that normal people find far to hard to use. Actually non normal shoe staring Moodle obsessives find it hard to use (I refer to myself). I spent a great deal of time trying to understand the code behind Cloze and it defeated me.

Over many years I have seen people roll their eyes in despair at this question type. Thank goodness (AKA TimHunt and team)  that core now has the OU fill in the blanks questions. I know that Cloze does much more than fill in the blanks but that is the main thing that people want.

There is also the lost opportunity cost. Time spent doing this could be spent on creating a nice ATTO plugin to edit Cloze questions to benefit so many more people. 


Marcus

Average of ratings: Useful (1)
In reply to Marcus Green

Re: CLOZE multichoice multiple answers

by Davo Smith -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

No one has offered to pay for my time to develop an Atto plugin, but they have already paid for my time to extend the Cloze question type to add an extra option that could be used if someone wants to (but, again, which has no impact at all on someone who doesn't use it). The level of complexity added to the code to support this new feature is very low (and it only took a few hours to develop so far), whereas the level of complexity of creating a new Atto plugin is much higher (although it does not impact on the complexity of the Cloze question itself, I would guess that it is 2-3 days of development time).

So, it is incorrect to suggest that making this change in any way takes away from time spent developing an Atto plugin that I'm not aware of anyone currently working on.


In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Sorry Davo, I get a bit unreasonable at the thought of extending the Cloze question type. 

" but they have already paid for my time to extend the Cloze question type"

Which is a very, very good reason to do it.

"it only took a few hours to develop so far"

You are a far cleverer man than I (but I already knew that)

"whereas the level of complexity of creating a new Atto plugin is much higher"

I am not so sure, every so often I take a look at the existing TinyMCE plugin and think that perhaps I should have a go at an Atto version. Then I remember the wise words of the great philosopher Clint Eastwood "A man should know his limitations"

A link to an old discussion about the  TinyMCE editor for those interested

https://moodle.org/mod/forum/discuss.php?d=221420





In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I think what is done here has to parallel what was doing for multiple choice, single response, with shuffled and unshuffled, and vertical/horizontal variants. (https://docs.moodle.org/31/en/Embedded_Answers_(Cloze)_question_type#Format)

I wonder if a better option would be MULTIRESPONSE, with all the variants like

  • multiple choice (MULTIRESPONSE or MR), represented as a dropdown menu in-line in the text,
  • multiple choice (MULTIRESPONSE_V or MRV), represented as a vertical column of radio buttons, or
  • multiple choice (MULTIRESPONSE_H or MRH), represented as a horizontal row of radio-buttons,
  • when the quiz question behavior shuffle option IS SET YES, the following multiple choice sub-questions elements will be shuffled,
  • multiple choice (MULTIRESPONSE_S or MRS), represented as a dropdown menu in-line in the text,
  • multiple choice (MULTIRESPONSE_VS or MRVS), represented as a vertical column of radio buttons, or
  • multiple choice (MULTIRESPONSE_HS or MRHS), represented as a horizontal row of radio-buttons.
In reply to Tim Hunt

Re: CLOZE multichoice multiple answers

by Davo Smith -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I can understand the vertical / horizontal + shuffled / not shuffled responses (although it is 'checkboxes' not 'radio buttons'), but I'm not sure how a dropdown menu would work.

I'd be tempted to default to vertical, then offer MULTIRESPONSE_H as a variation, so the list would be:

  • MULTIRESPONSE or MR - vertical list of checkboxes, not shuffled
  • MULTIRESPONSE_H or MRH - horizontal list of checkboxes, not shuffled
  • MULTIRESPONSE_S or MRS - vertical list of checkboxes, shuffled
  • MULTIRESPONSE_HS or MRHS - horizontal list of checkboxes, shuffled


In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

That seems like a plausible list. Would be good to hear from some people who actually use this question type.

In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Daniel Thies -
Picture of Core developers Picture of Plugin developers Picture of Testers

Davo,

The scoring is more complex than the single answer case. The sum of correct (positive) answers must equal 100%, and the parser should check that. Wrong answers receive negative marks. Otherwise one could check them all. If = or ~ are used, they should default to the pattern of oumultichoice.

When I have some free time I will do the Atto plugin.

Average of ratings: Useful (1)
In reply to Daniel Thies

Re: CLOZE multichoice multiple answers

by Davo Smith -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I've already addressed these both in the tracker ticket:

"Note that to simplify the syntax, correct answers are automatically scaled to give a total of 100% (so, if there are 4 correct answers, each is automatically worth 0.25) - if scores are manually given that add up to exactly 100%, then no scaling is done. Negative scores are not scaled. If any answers are given a score of 0, then selecting them makes no difference to the final score."


In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Daniel Thies -
Picture of Core developers Picture of Plugin developers Picture of Testers

I am sorry i misread your example above and missed the tracker comment. However, I think that it is still an area which could cause confusion for users. It would not be obvious to most why the scaling happens if they put in explicit positive percentages.

In reply to Daniel Thies

Re: CLOZE multichoice multiple answers

by Davo Smith -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I'd counter by suggesting it is not obvious why it should be a teacher's responsibility to make sure that all the positive answers add up to 100%, when that is a job that can be done by a computer far more easily.

Having set up many multichoice, multiple answer questions during my teaching career, I cannot recall any time that I really relished the opportunity to figure out that 100/7 = 14.29%, so that I could allocate each correct answer that % (or, subsequently, work out what 100/8 was, just so I could add 1 more 'correct' answer). In every case - what I really wanted was 'all of these answers are correct ...' and 'all of these answers are wrong ...' and have the computer figure out the numbers (and I'm a fairly mathematical, IT person).

Maybe I should go further and make it so that any 'incorrect' answers are automatically given a negative score, so that most teachers never have to enter % at all for this question type (within Cloze) ...

In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Daniel Thies -
Picture of Core developers Picture of Plugin developers Picture of Testers

Yes, I agree. I think that what you have done looks good, but it is the syntax of cloze that is confusing to start with. The multiple correct answers adds an additional level of complication to it. It would be great if teachers could get a working question without entering %. However, I would imagine some teachers would also like the total penalties to exceed one.


In reply to Daniel Thies

Re: CLOZE multichoice multiple answers

by Davo Smith -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

There is no scaling on the 'incorrect' answers. I was just wondering if a teacher didn't mark an answer as 'correct' and didn't give it a specific %, maybe it should just be given a fixed penalty (-50%) or a scaled penalty (so that incorrect answers all add up to -100%, if no penalties are specified)? This would not stop a teacher who wanted to specify the +ve or -ve scores for each answer, but would greatly simplify the question type for those who didn't want to do so.


In reply to Daniel Thies

Re: CLOZE multichoice multiple answers

by Daniel Thies -
Picture of Core developers Picture of Plugin developers Picture of Testers

It would seem logical to me that there should be a default penalty of some type. Do your customers have a specification for that?

After thinking about my own usage of embedded answers question, I realize that I only really use it when no other question type will work well. In those cases I usually do not care that the syntax is difficult. I also do not use multiple response that often, but I imagine there are good reasons to do so.

I did start on an Atto editor plugin for cloze, but I am not sure how useful it will be.

In reply to Daniel Thies

Re: CLOZE multichoice multiple answers

by Germán Valero -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

The new cloze editor for Atto plugin that Daniel Thies has been writing looks very good and useful. I will test it more fully and write back.

In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Davo Smith -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Based on the discussion here, I've updated the ticket with the following changes (text copied from the ticket):

Now called 'MULTIRESPONSE'.
Horizontal and shuffled options added (MULTIRESPONSE_H, MULTIRESPONSE_S, MULTIRESPONSE_HS).
Positive fractions (whereby '=' is treated as 100%) are scaled so that the combined total of all positive fractions is 100% (nothing happens if they already add up to 100%).
If no fractions are specified at all, then all incorrect answers are given a fixed score of -50% (to avoid having to use fractions at all in the most simple version of the question), if any fractions are specified (either positive or negative), then incorrect answers will need to be manually given a negative score.

For clarity, a couple of examples:

Please select the fruits {1:MULTIRESPONSE:=Apple~Burger~Hot dog~Pizza~=Orange~=Banana} and vegetables {1:MULTIRESPONSE_HS:%10%Raddish#Good~%-20%Chocolate~%-30%Biscuit~%-10%Cheese~=Carrot}

This translates into:

  • Multichoice, multiple answer, unshuffled, vertical checkboxes
    • Apple +33.3% (scaled so all +ve answers add up to 100%)
    • Burger -50% (automatic negative score for answers not marked as correct)
    • Hot dog -50%
    • Pizza -50%
    • Orange +33.3%
    • Banana +33.3%
  • Multichoice, multiple answer, shuffled, horizontal checkboxes
    • Raddish +9.1% (correct answers of +10% and +100% scaled to 9% + 91%, to add up to 100%)
    • Chocolate -20%
    • Biscuit -30%
    • Cheese -10%
    • Carrot +90.9%

So, the simple version, where no percentages are specified, works as expected - i.e. select all (and only) right answers = full marks, any wrong answers result in less than full marks. At the same time, the teacher still has the option of manually specifying percentages, at which point they have the flexibility of specifying the exact negative marks for incorrect answers (including the option of specifying a mark of 0 for any answer).

Further feedback (especially from trying out the question type) would be appreciated.

In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

That sounds good.

Do you support the abbreviation MR (like MC for MULTICHOICE)?

I am not sure that -50% is best. That seems arbitrary. I think the two most common options are -100% (any wrong answer and you lose) or -100/(num right)% (so 33.3% in your example) so one wrong choice selected cancels one right choice. I would make that last option the default.

In reply to Tim Hunt

Re: CLOZE multichoice multiple answers

by Davo Smith -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Yes - MR, MRH, MRS, MRHS are all supported (but not MRSH).

I agree that defaulting to -100/(num right)% sounds about right, so I've set that as the default in the latest commit.

That means the first example above now gives:

  • Multichoice, multiple answer, unshuffled, vertical checkboxes
    • Apple +33.3% (scaled so all +ve answers add up to 100%)
    • Burger -33.3% (negative version of correct score - not affected by the number of incorrect answers)
    • Hot dog -33.3%
    • Pizza -33.3%
    • Orange +33.3%
    • Banana +33.3%


Average of ratings: Useful (2)
In reply to Davo Smith

Re: CLOZE multichoice multiple answers

by Sarra Lev -

Sorry if this is repetitive, I'm never really sure where to post - but how about this, instead of negative scores for choosing the wrong answers:

Question: Canada is:

Choice #1: North of the US - (20%)

Choice #2: A country - (20%)

Choice #3: Cold all year-round - (20% if they don't choose this)

Choice #4: More populated than the US - (20% if they don't choose this)

Choice #5: A monarchy - (20% if they don't choose this)

Etc.