Trying to use cloze for programming quiz

Trying to use cloze for programming quiz

by Dov Kruger -
Number of replies: 12

Greetings fellow Moodlers.

I am using moodle 1.9 and 2.6.2, trying to create questions in cloze to test students programming in C++ and Java.

First of all, can anyone tell me whether cloze itself is the same between the two versions?

I write the following code, and it tells me that it is not case sensitive.  But I need it to be.  I cannot find in the documentation how to require that the words be typed as I want them.  For example, "new" is a keyword in C++ and Java, and cannot be uppercase.

The second question is about regular expressions within cloze.  I need pattern matching because although the answer: rows*cols

is good, so is rows * cols.  In fact, any number of spaces is fine.  I would rather use regex.  The only reference I found in the documentation was this: https://moodle.org/mod/data/view.php?d=13&rid=338

The problem is, there are no examples, it describes what to do in 1.9, but not what happens in 2.6.2.  I am using 1.9 at the college where I work, but setting up 2.6.2 myself, and I don't want to do anything that does not work in both versions.

class Matrix \{
private:
  double* m;
  int rows;
  int cols;
public:
  Matrix(int rows, int cols) \{
    m = {1:SHORTANSWER:=new} double[ {1:SHORTANSWER:=rows*cols}];
    {1:SHORTANSWER:= this->}rows = rows;
    {1:SHORTANSWER:= this->}rows = rows;
  \}
\};

 

Average of ratings: -
In reply to Dov Kruger

Re: Using Cloze questions to test programming skills

by Mary Cooch -
Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

Hello there. This is not my area but does the documentation for the regular expression short answer version help http://docs.moodle.org/en/Regular_Expression_Short-Answer_question_type and its entry in the new plugins database? https://moodle.org/plugins/view.php?plugin=qtype_regexp ? I don't know but I'd have thought the syntax was the same for 1.9 as for 2.6

Average of ratings: -
In reply to Mary Cooch

Re: Using Cloze questions to test programming skills

by Dov Kruger -

One of the highest priority items I could suggest to this forum is the ability of the author to delete his/her own posts.  There is no reason why it should be frozen in stone after 30 minutes.  In this case, I discovered multiple things I would have liked to change about this post.

First of all, I am not using regex at the moment, though I would eventually like to.  But if I am dealing with a 1.9 implementation and don't control it, it's not clear to me that regex will work.

I found bugs in the 1.9 implementation of CLOZE and wanted to report them, only to find that 1.9 is no longer supported  The bug does not exist in 2.6.2, so presumably someone found it anyway.

Thanks for answering, and sorry to waste your time in this instance.  I tried to get rid of this, I really did.

The documentation of the regex plugin is kind of minimal.  It doesn't tell a programmer what is supported, and doesn't tell a beginner how to use it properly.  I will try to fix that, eventually.

Average of ratings: -
In reply to Dov Kruger

Re: Using Cloze questions to test programming skills

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

"One of the highest priority items I could suggest to this forum is the ability of the author to delete his/her own posts", 

You are not the first and won't be the last to notice that bit of peculiarity. I have written little hacks in the past to make that whole set in stone thing go away on my own Moodles but it is unlikely to ever go away at the mother ship.

Note that if you are creating questions on programming languages that are answered using reg ex expressions you are going to spend quite a bit of time escaping the Math/Programming symbols that are also reg ex symbols, i.e.  [ ] {}  * . / \ etc etc so they are treated literally rather than as reg ex symbols. I spent time writing a "turn off regex matching" mode with my question type specifically to make it easier for it to be used with Math/Programming questions. (Which I now notice in your example and so I am glad I re-read it before 30 minutes was up (grins))

 

 

Average of ratings: -
In reply to Dov Kruger

Re: Using Cloze questions to test programming skills

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Dov,

I can't let you write that the REGEXP plugin question type documentation is "minimal".sad I have spent many hours on making the documentation explicit and user-friendly. Of course, it can always be made better. Have you actually read it in full at Regular_Expression_Short-Answer_question_type?

What you want to achieve with your Cloze question is quite possible by using the REGEXP question type embedded in a Cloze question.

class Matrix {
private:
  double* m;
  int rows;
  int cols;
public:
  Matrix(int rows, int cols) \{
    m = {1:REGEXP_C:=new} double[ {1:REGEXP_C:=rows*cols~=rows(| )\*(| )cols}];
    {1:SHORTANSWER:= this->}rows = rows;
    {1:SHORTANSWER:= this->}rows = rows;
  }
};

will accept "new" but not "NEW", and it will accept "rows * cols" as well as "rows*cols", as shown on this screenshot

screenshot #1

I'll be pleased to answer any further queries re REGEXP and hope you will find it useful. Please note that I no longer maintain it for Moodle version prior to 2.x.

Joseph

Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Using Cloze questions to test programming skills

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

MS Word counts your page at a little over 4,000 words which is significantly more than the core Cloze type.

Average of ratings: -
In reply to Joseph Rézeau

Re: Using Cloze questions to test programming skills

by Dov Kruger -

How about if I just lay out my questions for you, and you can decide whether your documentation answered these questions.
I'm going to assume that you are only documenting for programmers who already know Regex, because if not, you should re-examine your statement about being user-friendly.  I will be happy to improve the documentation from the point of view of someone who doesn't know the answers once I learn enough to make sense.

Is regexp built into 2.6 or do I have to add a  module to make it work?

You don't list out what is supported methodically.  Can I assume it is the perl standard Regex? That's the one I know.  Or is it PHP?

Instead you state the operators, and not what they do.

[aeiou] one of the letters

[a-z]    range

\s   shorthand for whitespace?

I assume ?*+ are all standard?

You use double brackets for permutation.  Very nifty and useful!

red_white_blue     this matches any permutation of red white blue?  

Does it match green red white blue orange? What about red white blue white?

Is there an intrinsic ^$ wrapped around every match? If so can it be defeated?

You wrote this as a solution for some of my code.  Why (| ) as opposed to " ?" (space questionmark)
{1:REGEXP_C:=rows*cols~=rows(| )\*(| )cols}

 

Average of ratings: -
In reply to Dov Kruger

Re: Using Cloze questions to test programming skills

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Dov,

Thanks for your reply, here are my answers to your questions.

1. "I'm going to assume that you are only documenting for programmers who already know Regex, because if not, you should re-examine your statement about being user-friendly."

Well, obviously a minimal knowledge of the Regular Expressions syntax is required in order to use my REGEXP question type. However I assume that most of the time those teachers creating questions will use only the most basic syntax, as illustrated by the examples I provide. For anyone wanting to know more about Regular Expressions on my help page I do provide links to 2 sites which I have found extremely useful. I think this is more useful than expanding my own help page with over-complicated details. But this can be debated.

I should also point out that this question type was primarily developed with language teachers in mind. Using it for questions about programming languages can work, but may strain its features beyond their limits.

2.- "Is regexp built into 2.6 or do I have to add a  module to make it work?"

It is clearly stated in the Help page that REGEXP is a 3rd-party add-on and a link if provided to the Moodle plugins repository download page.

3.- "You don't list out what is supported methodically.  Can I assume it is the perl standard Regex? That's the one I know.  Or is it PHP?"

It's the PHP regular expressions. As for a full list of supported syntax, I refer the user to the links referred to in my point #1 above.

4.- "you state the operators, and not what they do"

Well, I do show what the operators "do", in my numerous examples. As is obvious from my Help page, I am not a programmer and I prefer to explain by examples, not theory.

5.- "[ [ red_white_blue] ]a)  this matches any permutation of red white blue?  b) Does it match green red white blue orange? c) What about red white blue white?"

a) Yes, as shown in the documentation!

b) Well, it would if you write the expression as needed it's [ [ _green_ _blue_ _white_ _red_ _orange_ ] ]

c) yes:

  • it's red white blue white
  • it's red white white blue
  • it's red blue white white
  • it's red blue white white
  • it's red white white blue
  • it's red white blue white
  • it's white blue white red
  • it's white blue red white
  • it's white white red blue
  • it's white white blue red
  • it's white red blue white
  • it's white red white blue
  • it's blue white red white
  • it's blue white white red
  • it's blue red white white
  • it's blue red white white
  • it's blue white white red
  • it's blue white red white
  • it's white red white blue
  • it's white red blue white
  • it's white white blue red
  • it's white white red blue
  • it's white blue red white
  • it's white blue white red

6.- "Is there an intrinsic ^$ wrapped around every match? If so can it be defeated?"

Yes there is. Why would you want it "defeated"?

7.-  Why (| ) as opposed to " ?"

Both are accepted regular expressions syntax and will work. As in "blue is my favo(|u)rite colou?r".

Do install the REGEXP plugin question type on your Moodle 2.6 installation and test it out. I'll be happy to answer any further query. But bear in mind that using it for questions about programming languages is not what it was meant for, it was only meant for processing natural languages.

Joseph

Average of ratings: -
In reply to Joseph Rézeau

Re: Using Cloze questions to test programming skills

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I am a programmer, I have written my a question type that uses regex, I have read through Josephs documentation very carefully a while ago (both to inform my question type and for other purposes)  and I came to the following conclusions.

Understanding regex is not trivial (even if you are a programmer) and it is complicated by the existence of more than one flavour.

If you are documenting a question type that uses regex, teaching how they work is way out of scope, there are shelves of books that already do that.

Joseph has included good documentation but as with just about all Free/Libre projects additional documentation (and examples) would be good and people are free to add to the Wiki. 

It might be useful if contributed question types included a backup (xml) file with sample question in to illustrate what can be done.

And finally Joseph is awesome because he gave me a great deal of help and advice when developing my question type.

Average of ratings: Useful (1)
In reply to Marcus Green

Re: Using Cloze questions to test programming skills

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Two things everyone should know about regular expressions:

'Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.' -- Jamie Zawinski. (See also this blog post.)

Average of ratings: Useful (1)
In reply to Joseph Rézeau

Re: Using Cloze questions to test programming skills

by Dov Kruger -

Thanks for your help Joseph.  You have to understand that I am implementing two graduate programming classes in moodle 1.9, so right now, testing out your code in 2.6.2 is not the highest priority.  I will get to it as the semester dies down.

You made a nice regex utility.  I think I can offer some changes to the documentation that would improve it, later.

For now, if you are trying to create a matcher for an answer, the last thing in the world you would want to do is require the user to type "it's" as a first word.  In fact, I would think anyone would want to ignore any such waste verbiage and focus on the core words.

You seem to have misunderstood my question.  I was asking whether your permutation operator would match surrounding words.

You seem to indicate that this:

[ [  _blue_ _white_ _red _ ] ]

turns into this:

^[ [  _blue_ _white_ _red _ ] ]$

which would be good for disallowing the user to add in other colors by guessing them all.

Of course, if you got rid of the anchors, the pattern would match:

it's red white blue!

it is red white and blue, final answer

but it would also match:

I think it's orange red white blue red green yellow pink

So there are advantages to allowing anything on either side, but also disadvantages

 

I personally would like a permutation list and a list of words that are incorrect, but that goes beyond regex into a hashmap.  But then again you have already done a permutation.  This is not a specific syntax recommendation, just highlighting what would be useful

[ [  _blue_ _white_ _red _ ] ]    :BADWORDS=orange,green.yellow,black

The reason I was surprised you use parentheses, aside from the fact I find them harder to see, is that they tend to be used to store values in regex engines.

I asked whether you support \s for space, and you didn't mention it?  Or in your simplified regex are you already turning every space into \s+

I would like to write things like:

p = new\s+int\s*[\s*rows\s*\*\s*cols\s*];

Since that's a little awkward to type, I would love some notation that simplifies it.  If a space means \s+ in your simplified version, I just need something simple that means "zero or more spaces". 

 

CLOZE is not the most convenient syntax to type, but I quickly banged out a web page that lets me edit text and quickly substitute specific values for CLOZE expressions.  It's a bit of a hack, but hopefully someday I can use it in moodle, it's way more convenient than the manual way i use now.  I am sure I could do the same type of front end for a regex so that I wouldn't have to type \s+ and \s* (or [ \t]+ and [ \t]* if you don't support \s).  What I need is access to the full regex, not a restricted subset. I will take another look at your documentation when I get the chance.

Average of ratings: -
In reply to Dov Kruger

Re: Using Cloze questions to test programming skills

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Dov,

1. "I personally would like a permutation list and a list of words that are incorrect, but that goes beyond regex into a hashmap.  But then again you have already done a permutation.  This is not a specific syntax recommendation, just highlighting what would be useful [ [  _blue_ _white_ _red _ ] ]    :BADWORDS=orange,green.yellow,black".

I do not see why you would want a BADWORDS list. All you have to do is to enter in an Answer box the following "(orange|green|yellow|black)" with a score value of None and an appropriate message to the student.

2. "I asked whether you support \s for space, and you didn't mention it?  Or in your simplified regex are you already turning every space into \s+"

My REGEXP engine removes all sequences of white spaces entered by the student and reduces them to one unique space. So I don't need the \s syntax.. In my analysis of "natural languages" syntax, there can never be more than one space between two consecutive words.

3. "CLOZE is not the most convenient syntax to type,"

Agreed, and you will find quite a few hacks proposing alternative ways to enter Cloze questions if you search this forum.

Finally, although my REGEXP question type will work as a sub-question of a Cloze question (see question/type/regexp#Inserting_RegExp_sub-questions_in_Cloze_type_questions), you lose a number of features of that question type which are available when using it as a stand-alone question (especially the Hint feature).

Joseph

Average of ratings: -
In reply to Joseph Rézeau

Re: Using Cloze questions to test programming skills

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

3. "CLOZE is not the most convenient syntax to type,"

I am at the front of the queue to agree with this (grins). But the syntax does delivers great power, and there is just about nothing else available free or paid for that gives you the flexibility of the Moodle core Cloze question type.

Average of ratings: -