Exam Module Implementation ideas

Exam Module Implementation ideas

by Paul Shew -
Number of replies: 25
A number of us are now using Moodle for exams, and John Bristor suggested that perhaps we need new "exam module" to deal with the specific needs of exams as opposed to simple quizzes. Using code from John, Timothy Takemoto is already looking into how to implement this.

Development of an exam module certainly sounds interesting. I'm starting this topic to spur discussion about what the needs of the community are, and how it could be implemented. There are a few ideas in the above referenced thread, but I think a bit more discussion would be helpful.

Here's some ways an exam module could be useful
From John Bristor: "every user is given a seperate key for each exam and is exclusive of other students. Emails to notify the user of their Key and emails sent to the teacher when a user started the exam and and when it ended with their score (also used for sending certificates if they passed)."

In another post, I raised the issue of data security (redundancy). It would be good to have the grade (and maybe even all of the student answers) emailed to the teacher upon completion of the exam. The redundancy provided by such a system would make it a more robust exam platform.

Here's a few questions for a discussion starter.

1. Why is an exam module necessary, seperate from the quiz module?

2. Could it not be created as an option within the quiz module? Isn't is essentially a difference in administration, with the same backend?

3. It seems like it would share questions, basic test format and a great deal of common functions/classes with the quiz module. How will they be integrated?

4. As a teacher, how else could an exam module be used, and what kind of features would you like? Think big! wide eyes
Average of ratings: -
In reply to Paul Shew

Exam as Secure-Quiz, Rat Maze, Auto-homework, Lure of Sirens?

by Timothy Takemoto -

Thanks Paul,

To be honest I now see that "exam module" may be an inappropriate appelation for what I had in mind. As you probably know, I have in mind the loathed and loathable automatic-homework-machine aka rat-maze - i.e. a quiz that is sufficiently secure to allow teachers to set it as homework to force their students to study extrinsically (as opposed to laudable intrinsic motivation methods).

In this respect I am mainly looking for a "more-secure-quiz" or "serious-quiz" and in that sense I agreed with John Bristor's "exam module" appelation. You have just done an exam which I presume you proctored. In this case your exam could be LESS secure than the quiz, with out even random questions for example. On the other hand, when I say "exam" I am thinking "Secure-Quiz," in the sense of increase cheat-proofness and seriousness.

By "seriousness" I mean to get away from the motivation for naming the "quiz" module as it is (something fun, not serious, ho ho) and get back to  a more brutal, draconian "do this or fail!" type, Japanese classroom atmosphere.  The addition of an extra level of passwording for instance is not essential to me but it does add a psychological dimension of seriousness as if to say "this exam is for you and you alone."

Precisely why you are interested in having mails sent to the instructor, I am not sure (please do tell) but I was able to spot cheating using some previous software through as a result of mails sent on completion. Some students would work together and press submit at the same time. Two identical results mails, appertaining to two students, would arrive in my inbox separated only by a matter or seconds. Again, psychologically, knowing that ones work is sent immediately to ones instructor is likely to add a non "just a quiz" more a "this is an exam" feel to things.

Of course it would be nice if the exams/quizzes are fun as well - hence my interest in examinable hotpotatoes and my classes themselves are really pally. It is partly because I want classes to be socially constructive, communicative and generally pally that I want to dispense the learner-whipping to a machine.

As to the question of whether this needs to be a new module or not, while I have said in other posts that it seems to me that there is no need for a seperate module, since the quiz is almost there...Upon reflection however, it seems to me that there are good reasons why the quiz is and has been almost there for a long time, and may be almost there for a long time to come. Particularly, since Martin is not keen on either the ethos (extrinsic motivation) or the implementation (often using javascript) it is no wonder that we have a bundles of hacks but still no "exam". The fact that the ethos and implementation of a possible exam module are different means that a different module might be a good idea, for the time being at least.

As for features, I would like ALL the hacks and ALL the features mentioned on Martin's list (of which there is fortunately considerable overlap) for starters. Is that big? Or just greedy?

So, if I may be allowed to turn the question back, how do you understand "exam module?" Is this something that you would proctor?

Tim

Timothy Takemoto

By the way, did I mention this? In my first and only class that I set Moodle tests every week, I managed to increase students average homework study time to about 2 hours! Automated homework was no "lure of the sirens" but an actuality. And as for the students? It would seem they are masochists. I recieved my first ever standing ovation at the end of the term! It is of course Martin (and associates) that deserve the applause, even if he did not mean to recieve it in this way. Indeed, it is probably with some horror since, this was a case of "plug em in and make 'em work"... that's my moodle motto!

Average of ratings: -
In reply to Timothy Takemoto

Re: Exam as Secure-Quiz

by Paul Shew -
To be honest, I think it should be the "exam option" within the quiz module. Having done a bit of work "under the hood" in the quiz module, I know that a tremendous amount of work has gone into creating it, and I think it's much easier and better to build on what we already have than to build something separately. But for the sake of brainstorming, it's good to think "outside the box."

I use quizzes to motivate my students to master the material and stay active in the class. I administered a quiz almost every other week last semester, and they were very well received by the students. It's funny, but I've noticed the same phenomenon you have -- I make my students work much harder than they do for their other classes, and they appreciate it in the end (at least most do). smile

Since all of the quizzes put together will have the same weight as the final exam, I want the exam to be different. It should be labeled "exam" or "test" and not quiz.

Furthermore, I want it to be very reliable. I want the emails as a backup. Just like many places with electronic voting machines still want paper printout "just to be safe," I want email of the test results immediately sent as an email for backup. What if the server crashes right after the exam and all the data is lost? It's a million-in-one-scenario, but I don't want to take the chance.

As for the anti-cheating hacks, I don't see how any of them can really work. The only effective anti-cheating mechanism is to proctor the exam on campus. If students are motivated enough, they will find a way to cheat. (I caught one student cheating on a *proctored* quiz last semester!surprise) On the other hand, I also recognize that the anti-cheating hacks will be effective deterrents for *many* students, and figure they can't hurt. In the last analysis, I think we have to say, "Testing is for your own benefit as a part of the learning process, so you're on your honor not to cheat." That's what I did for my quizzes, and I think it worked well. The final exam was the test to see if they really learned the material or not.

But disabling things like the back button or copy function during the exam (or even removing the header links) would be a good thing to impress upon the students the seriousness of the exam.

To summarize my top two "exam option" features are:
1. seriousness (naming, format, etc.)
2. robustness (redundant storage of answers/results)

I think that "cheat-deterrents" are also a good thing and many other Moodle teachers are probably looking for them. Furthermore if people are doing online-only classes, then they will probably have a whole different set of exam needs.
Average of ratings: -
In reply to Paul Shew

Re: Exam Module Implementation ideas

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Here's a few questions for a discussion starter.

1. Why is an exam module necessary, seperate from the quiz module?

MG:
 like the idea of a single exam that allows you to be given questions on differing topics within a subject. I probably like this because that is what I have implemented in http://www.jchq.net/phezam which is based on the approach taken by the prometric style exam. My system shows a graph style thing that shows how well you did on each topic.

If anyone wants to see my ugly PHP source let me know.



2. Could it not be created as an option within the quiz module? Isn't is essentially a difference in administration, with the same backend?

MG:
Perhaps... One of the hacks being spoken about is the idea of an option to allow a "question per page" approach. Having done that in my thingy (OK phezam from here on in), I have been looking at the quiz code and that feature certainly seems doable. As for the multiple topics, I'm not so sure. The current quizzer was designed to test on the "current" topic so it would require some banging at with a sharp instrument.

3. It seems like it would share questions, basic test format and a great deal of common functions/classes with the quiz module. How will they be integrated?

MG
Exactly, why re-invent the wheel when the interface to the current quiz seems so ssssexy. The backend interface to my Phezam is as ugly as an ugly thing and it only does multi choice questions.

4. As a teacher, how else could an exam module be used, and what kind of features would you like? Think big! wide eyes

MG
OK you really got me, you got me so I can't seleep at night....

Select how many questions per page on the front end is my thang...

But on the back end...

How about a keyword marking system for longer text questions. Thus when you set the exam you enter the keywords you expect in the answers. Then when hand grading (gotta happen on some tests), those keywords show up in red so marking is quicker...

How about a collaborative thing for cross marking? a voting system perhaps, allow other folks to put in suggestions for keywords.

How about a mindless spell checker for long text questions that parses the text via a word list and highlights mis-spelt words. No context or hyphenation checking just a siimple comparison.

How about a fork of the current quiz module instead of an attempt at replacement, call it the exam module and concentrate on the more exammy type stuff than the shorter quizzy stuff.

OK, OK so I  am using imprecise language but thems my thoughts.

Marcus

Average of ratings: -
In reply to Marcus Green

Re: Exam Module Implementation ideas

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I forgot to mention the other big difference between a quiz and an exam is length. The standard prometric type exams have around 60 questions and a single page with 60 questions on it is very cumbersome to deal with.

Average of ratings: -
In reply to Marcus Green

Re: Exam Module Implementation ideas

by Grant McWilliams -
Some of my Prometric tests are closer to 100 questions. When you add in the pictures that a lot of them have this just isn't going to work. Also another thing I just ran into this last week was that if the exam is too long and is on one page then the session times out before they are done. When you have a whole list of subnetting scenario questions the students take a long time. If the questions were on seperate pages the server would reload between questions... I upped my session timout to 4 hours but that isn't the solution.

Grant
Average of ratings: -
In reply to Grant McWilliams

Re: Exam Module Implementation ideas

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Grant said.....

"Also another thing I just ran into this last week was that if the exam is too long and is on one page then the session times out before they are done"

Another good reason to want a one question per page options. Now here is one for the developers, if you are going to have the user scrolling Next and Previous through the pages before submitting them, how are you going to store their answers? The two ways I can see is either in a session or by writing to the database as "unsumbitted answers". Storing in the session might run the chance of overloading the PHP session which is really not designed for storing significant amounts of data I believe. However it does seem to work OK for my multi choice 60 question mock exams with my phezam

(http://www.jchq.net/phezam)...

However if I were to hack the moodle quiz I would have to cope with longer question formats and the data requirement would go up. Are my assumptions about the limitations of a PHP session reasonable, and does it mean I ought to be looking at writing to the database as "unsumbitted answers"???

Marcus

Average of ratings: -
In reply to Marcus Green

Re: Exam Module Implementation ideas

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
In the database, even if it's in a special table for the purpose ...
Average of ratings: -
In reply to Marcus Green

A messy way of implementing multiple page tests.

by Timothy Takemoto -

I wonder if the description questions could be used in some way to make page breaks, so that, like in a pencil and paper exam, the examinees are given all the pages of the exam at the start. Then there would be no need to create a special new database table. 
If there were a cloned, changed version of the "description questions" with some javascript inside them, they might be able spawn a new window containing the rest of the exam.
The submit button in the last window might be able to submit all the other windows, I guess. http://p2p.wrox.com/archive/javascript/2003-02/12.asp
Very messy.

Average of ratings: -
In reply to Timothy Takemoto

Re: A messy way of implementing multiple page tests.

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I have an instinctive aversion to JavaScript so I have been looking at how most of a "question per page" solution can be done on the server side. The current "save your attempt" button triggers a popup to say something like "do you really want to submit this". In my modified version I have a pervious and next button so I need to work out which button was submitted before deciding to go to the next/previous page or if it was a subit these questions choice. One way to do this is through a javascript function sad.... Perhaps someone can suggest a better server side way.

If however I do it in JavaScript it seems that the javascript headers are included in a standard include file that probably should not include me messing with it.

My Question

Should I try to do it all on the server or should I do it in JavaScript, and if I do it in javascript should I modify javascript.js or put it in the body of the page.

Also is this the right place to ask a question like this and is there anyone else bashing away at this issue as It would ba a pity to duplicate such work.

Marcus

Average of ratings: -
In reply to Marcus Green

Re: A messy way of implementing multiple page tests.

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Server-side is far better, and the less Javascript the better (too much Javascript is the mistake WebCT made). You can use a new table for storing the data so far and collecting it up, controlled by one new script which is like attempt.php but does the work of chopping quizzes into parts and storing the answers from each part.

Then on the final submit this script pulls all the data out that it has collected, joins it together and replicates the normal logic that you see today in attempt.php to record the attempt.
Average of ratings: -
In reply to Martin Dougiamas

Re: A messy way of implementing multiple page tests.

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

You have confirmed my most deeply held predjudice, don't use JavaScript unless it is unavoidable, and it is almsot always avoidable.

Marcus

Average of ratings: -
In reply to Marcus Green

Re: A messy way of implementing multiple page tests.

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
I feel the same way about Java.  wink
Average of ratings: -
In reply to Martin Dougiamas

Re: A messy way of implementing multiple page tests.

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Mr Moodle, go to google and type in the following keywords

Marcus Green Java

Now wash your mouth out with carbolic soap and water (easy on the water).....

wink

Average of ratings: -
In reply to Marcus Green

Re: A messy way of implementing multiple page tests.

by Przemyslaw Stencel -
Marcus, I see you're working on this feature. I think it would be a very useful enhancement of the quiz module. Can you tell me how exactly you are going to implement this? Will this simply be a choice between all questions on one page and one question per page? Or will it also be possible to create quizzes where there is any number of questions per page? If so, will it be possible to have a different number of questions on the different pages of a given quiz?

Wow, that's a lot of questions wink
Cheers,
Przemek
Average of ratings: -
In reply to Przemyslaw Stencel

Re: A messy way of implementing multiple page tests.

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I certainly intend to put some time into this, but don't expect any code in the near future as I am a rather slow worker.

My original idea was a boolean choice between all questions on one page or one question per page. Since then I have realised there is a desire to select the number of questions per page.  I am working on the assumption that the number of questions per quiz would be set up with the quiz. Indeedy I think I have modified the database on my system to have such a column though I am not using it at all at the moment.

As of last night I had got as far as modifying the quiz page so it had a Previous Next button, was dumping the a digit indicating the number of questions in the quiz and when I pressed the previous and next button it did nothing but then again it didn't break anything either (thus I was re-feeding the current quiz id back to the page in the form info).

A few months ago I also had a session variable that was tracking the "theoretical" current page, just so I could see how that fitted into the moodle code way of doing things. My background is in Java (see previous Martin tease) so I am getting used to the slightly more procedural approach of Martins code.

A journey of a million miles starts with one step and so far I have just shuffled one heel a little.

Marcus

Average of ratings: -
In reply to Marcus Green

Re: A messy way of implementing multiple page tests.

by Przemyslaw Stencel -
That's good you've only shuffled one heel a little and not even made the first step, because I'd like to ask you to consider changing the direction of your 1.000.000-mile journey. wink

Would you be able (and willing) to design it in such a way that it is possible to chop the quiz into unequal chunks? From my experience, I have often wished I could divide a quiz into pages where questions which are similar or fall into one set of instructions could be grouped together (and these groups were not equal in terms of the number of questions).

In practice, instead of a quiz setting with the number of questions per page, this could be done in one of these ways (these are just suggestions, there may be better ways):
  1. Create a "page break" question type. Similarly to "description", this would not be a real question - it would only be used to mark where the quiz should be split.
  2. Have a "create page breaks" in the left panel of the quiz editing screen. Hitting the button would reload the page with checkboxes labelled "page break here" between all questions. Optionally, there might be a "select all page breaks" button which would trigger a javascript (!wink) to check all boxes.
Personally, I think solution 2 is better, because it doesn't create meaningless entries in the questions pool.

I'm not really sure I've managed to explain this well enough. If not, let me know and I will create mock interfaces to illustrate the idea.

Przemek
Average of ratings: -
In reply to Przemyslaw Stencel

Re: A messy way of implementing multiple page tests.

by Hans de Zwart -
I have seen this in some survey software:

You have surveys, groups and questions.

In Moodle this would translate to:

You have quizzes, groups and questions.

All question belong to a single group.

Then for display options there would be three choices:
  1. One question per page
  2. One group per page
  3. One page per quiz
Average of ratings: -
In reply to Przemyslaw Stencel

Re: A messy way of implementing multiple page tests.

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I think my short answer is no and my long answer is that my motivation is partly driven by my desire to make it easy to deliver long exam type quizes that have 3 0 - 60 questions. Although I am just twiddling my toe with this in moodle I have developed similar functionality with PHP and mysql in the past so I feel like I can do it.

However do show me an interface idea because I would rather conside all the design options before I start to shuffle rather than find myself walking around in a very small circle. (Have we taken this analogy to breaking point?)..

Marcus

Average of ratings: -
In reply to Marcus Green

Re: A messy way of implementing multiple page tests.

by Przemyslaw Stencel -
Hi Marcus,

You wrote: my motivation is partly driven by my desire to make it easy to deliver long exam type quizes that have 30 - 60 questions
Actually, doing it the way I suggest will allow exactly this, but in a more flexible way - not just all questions on one page or one question per page, but it will allow to have any number of pages and any number of questions on a page.

The attached image shows an example quiz that should be chopped into two pages, each beginning with instructions followed by a few questions (5 on 1st page, 3 on 2nd page).
Attachment pagebreaks1.gif
Average of ratings: Useful (1)
In reply to Przemyslaw Stencel

Re: A messy way of implementing multiple page tests.

by Steven Geggie -
Can I get a copy of your PHP code for this?  (I'm also interested in making sure students cannot print/copy/paste)
Average of ratings: -
In reply to Marcus Green

Re: A messy way of implementing multiple page tests.

by Przemyslaw Stencel -
When you click on the "create page breaks" button, the page is reloaded (see screenshot) and you can chop the quiz into as many parts as you want. For convenience, the "select all page breaks" button checks all boxes, thus creating a "1 question per page" quiz.

What do you think about it?
Attachment pagebreaks2.gif
Average of ratings: Useful (1)
In reply to Przemyslaw Stencel

Re: A messy way of implementing multiple page tests.

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
I will limit my (current) ambitions to creating code that allows one question per page. This will allow the state of each question to be saved when the user moves to another.

Once that is created those other permutations such as having a page break question type become viable as they will depend on the ability to save the state of the question response. My experience of software development is that the greatest key to success is to limit ambitions.

Average of ratings: -
In reply to Przemyslaw Stencel

Re: A messy way of implementing multiple page tests.

by Ger Tielemans -

Some of my teachers would find this an elegant solution of their problems. If I could vote for this I would vote... (Although I personally still prefer the essay-question)

Average of ratings: -
In reply to Marcus Green

Re: Exam Module Implementation ideas

by Don Hinkelman -
Picture of Particularly helpful Moodlers Picture of Plugin developers
I use the quiz module now for once-a-semester 50 question exams. This is done in a computer classroom during a scheduled exam hour. I supervise the whole room. I have had no problems and all 70 students left the room positive (My post test interviews seemed to indicate they like online exams better than paper ones).

One minor technical problem is that when students switch encoding midway, all previous answered questions are wiped out and they (three students) had to begin again. I think this is an Apache setting problem I can handle. but it would be nice if Moodle did not wipe out the answers (this refers to version 1.1.1)

All questions were previously used questions from the quiz database, plus two essay questions given via the Journal tool.

I had no session time out problems noted by others.

My main need an exam/modified quiz module is to have:
- a new interface for viewing, selecting, and sorting 50 or more questions. Unless you give very detailing item titles, it is hard to see the questions without opening them up. Also moving a question from the bottom of the list to the top of the list is incredibly time consuming. We need something drag and drop.
- question grouping: if I do question shuffling then I need to fix groups of questions together (multiple questions follow a single reading or picture or audio)

Absolutely it should be an addition to the quiz module because I need to do the detailed statistics available there. We analyze items because this is used as a foreign language placement test for incoming freshmen. We redo the questions each year according to item facility and item discrimination results.


Average of ratings: -
In reply to Don Hinkelman

Re: Exam Module Implementation ideas

by Biso Shrestha -

My school wants a similar thing to what you are currently doing...  An electronic, suprivised quiz/exam room.  Because you are writing in a suprvised environment you can also control the client machines as well.  To this end, isn't it possible to bypass many of the problems with cheating by using a stripped down browser?

My thoughts in this matter is maybe a VB-based version of IE that is just the HTML renderer.  It would read the address of the moodle site off some config file then goes directly the the site.  There would be no menus, no navigation buttons, only the ability to navigate within the window.  The browser would have to be passworded in order to quit, and prevent the use of the program manager, as well as alt-tab.  Also, the right-click menu would need to be disabled.

Would this be a better solution for some kind of exam room?

Also is it possible to make the session length equal to the exam length to solve the timeout problem?  If not, why not just simply have moodle automatically make a new page after say 5-6 questions, so you don't have to waste time on assigning page breaks?

Average of ratings: -