Quiz UI development

Quiz UI development

by Olli Savolainen -
Number of replies: 26
At the university of Tampere we are developing an electronic exam application around moodle. The basics (reservation via internet to an exam done in a surveilled class) are ready. However, in testing it has become clear that the UI for creating and managing quizes needs improvement.

Since we have at least around hundred teachers we need to have learn to create exams, we cannot possibly teach the quirks of the current UI. We will be very interested in cooperating the moodle community in working on it during this Spring and Summer.

For now I would be very interested in knowing what are the current plans to develop quiz for 1.8 and 1.9? Also, where and in your development cycle I could possibly plug in to do usability testing and planning? I'd be interested in any related documentation and discussion, though I have skimmed http://docs.moodle.org/en/mod/quiz/grading and wandered around in http://docs.moodle.org/en/Developer_documentation. We already have a rough list of what is the most hard to teachers - we are still working on it (through user testing) though, but I'll post it also here as soon as we get it just a bit more ready.

I am a professional PHP guy myself and also a student here in HCI/usability and have been integrating our e-exam app to moodle since last June.

Thanks smile
Average of ratings: -
In reply to Olli Savolainen

Re: Quiz UI development

by Don Hinkelman -
Picture of Particularly helpful Moodlers Picture of Plugin developers
Hi Olli,

Tim Hunt of Open University UK will be the key responsible one to answer your questions. I would like to say we at Sapporo Gakuin University will be glad to help with comments or field testing if you need that. We were involved early on to use Moodle for large scale testing with simultaneous classes of students. From four years ago, we helped fund development of the flash mp3 player and recently other enhancements, including a split screen mode for quiz media. Any UI efforts are greatly appreciated--especially ways to simplify for new students and new teachers. Looking forward to seeing your "rough list" when it is ready.

Cheers, Don
Average of ratings: -
In reply to Don Hinkelman

Re: Quiz UI development

by Olli Savolainen -
Sorry for the delay in answering. Actually, my resources are rather spare as well, I'm working part time 12h/week until summer, which will hopefully be full time and I'll have time to actually dig into things. Anyway, I was excited to get such supportive replies from you all. Thank you!

Don: as you can see from what's below, it will still need to be verified what's really critical, and of course if you already know about something that's problematic with the quiz UI (and this possibly verified through testing), that would be great. Once we get some prototypes, we'll be grateful for any sort of testing done on those, for example paper prototyping. What do you think?

Tim: thank you for letting me know the status of your work. Hope we'll be able to help you with your workload smile.

The main problems reported with quiz and new teachers by our support team have been as follows. However, we will still need further testing during the spring to actually know what are the most critical problems. Just to try to keep you up to date about what's going on here, I'll give you some of the thoughts we have so far. Hope I'm not being too aggressive with the following!

Question categories

The concept "question categories" versus the questions which are actually already in the quiz. Since we here are developing an exam system which is supposed to partially replace conventional paper exams, the natural mindset for teachers is: to create an exam you design questions and write them into the exam.

In contrast, moodle's current model requires the teacher to first create the questions into a category and then add them to the exam, which has been reported to be very counterintuitive. They just want to put the questions directly into the exam (Nielsen's heuristics: Flexibility and efficiency of use); the fact that they are at the same time creating them in a default category could parhaps be hidden in an "advanced" screen. The system of concepts that needs to be understood isn't really communicated to users who don't already know how it works.

Even if they could grasp the idea, there's simply too much going on in the quiz building screen (including the subtabs). Most teachers don't want to bother themselves with sharing questions; however, many of the exams we will have will have multiple essay questions randomized from categories (which is experimental functionality it seems). This is something most teachers will want, but with the current UI, have no idea of how to create. We have way too many (new) teachers to teach all of them to use the current UI for that.

I'm thinking about initially supporting only the simplest of operations (see progressive disclosure), and then having much of the current things behind an "advanced" button. Anyway, this is just the first thought, completely untested. Since managing questions and categories already has a separate screen of its own, it would seem simpler to keep those tasks away from the quiz UI, but this is problematic of course since the tasks really are related.

Tabs

The current hierarchical tabs in quiz management are rather confusing. If you have no questions you will be thrown to the edit tab no matter what, but there's no feedback about why that happens even though all the other tabs are active (links), too.

The active tab isn't really highlighted so it's hard for users to even know where they are; furthermore, the whole idea of "hierarchical tabs" seems problematic, and would probably be best replaced with a hierarchical menu on the left side of the screen (see for example winamp's or eclipse's config windows). Or perhaps the fact that there is a hierarchy could be made more visible (see, for example, firefox 1.5's - not sure about ff2 - tools -> options window). I'm not even sure that a hierarchical menu structure is really required at all for quiz, but naturally redesigning the entire UI structure is something that will take more time than just the half hour I have right now to try to continue the discussion. smile


That is my 5 cents for now. Looking forward to hearing your thoughts about all this.
Average of ratings: -
In reply to Olli Savolainen

Re: Quiz UI development

by John Isner -
I agree that most teachers don't understand categories. Unfortunately, categories can be ignored only in the most trivial scenarios, such as your example of the teacher who "just wants to put the questions into an exam."

Consider a PC file system. Should new users be encouraged to keep all their files in the one big directory (by hiding the hierarchy), or should they be encouraged to think about organizing their files hierarchically from the beginning, to avoid headaches later?

Given our lack ability to search questions in Moodle, hierarchical organization is even more important for questions than for files. Perhaps the problem of teachers who do not understand categories could be solved, not by hiding the categories, but by making them more obvious through improved user interface design, better documentation, etc. (Personally, I like the current design!).
Average of ratings: -
In reply to John Isner

Re: Quiz UI development

by Olli Savolainen -
I see your point. You might be right in that hiding the entire functionality may not be the way to go - it can be made a recommendation that teachers free-willingly choose to "do it right" in the first place.

Anyway, this is a "philosophical" choice: in order to market our electronic exam system here (in the University of Tampere), we need to ensure that teachers don't have to make things more complicated than is necessary at any given moment, since there are a lot of them: any potential complexity is a risk for us that the service might not be taken into use as widely as we'd like. Teaching them is not always a choice we can make.

It is important to allow teachers, who think they simply will not have more than one exam, to keep it simple. The UI should definitely allow them to start organizing things from a default category, in case they later realize that they need to organize questions.
Average of ratings: -
In reply to Olli Savolainen

Re: Quiz UI development

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Any time you can spend working on an impoved interface would be great. And doing paper prototypes, then inviting comments would be a great way to proceed. A good public place to post documentation about proposed new features is on a page linked to http://docs.moodle.org/en/Developer_notes on the documentation wiki.

And it is really helpful to have your observations about what currently is causing problems for your teachers.

In Moodle 1.8, the design of the tabs has been changed, which makes things much clearer.

Perhaps a minimal change to make the current interface clearer is just to add "This quiz" and "Question bank" headings to the left and right of the screen. What do you think?
Average of ratings: -
In reply to Tim Hunt

Re: Quiz UI development

by Olli Savolainen -
(I'm looking at 1.6)

smile Indeed headings titles probably are already an improvement. However, the box with the category selection would also seem problematic to me; as a minimal solution still, it might be a good idea to clearly make the questions box look subordinate to the category, since I guess that's what they are. Correct me if I'm wrong. Below, a quick GIMP sketch about how to perhaps do it.

Good to hear that the tabs will be fixed. However this is problematic for us, since we've made quite many changes to 1.6 so upgrading will probably be painful. Of course there's not much you can do to help this. In an "ideal world", our electronic exam functionality could perhaps be made an alternate "mode" or something of moodle core, but I guess there are lots of barricades to comfront before really imagining this to be possible.


Attachment mdl163-quiz.png
Average of ratings: -
In reply to Olli Savolainen

Re: Quiz UI development

by Olli Savolainen -
Hi again,

I have now started to design a new UI for quiz from the ground up, which focuses on the current exam being made and thus simplifies the process a lot for teachers who are beginners with moodle (or a keyboard and a mouse ;) ). I do see the new UI also a potential replacement for the current quiz UI, or at least an optional add-on of some kind: it can support all of the features of the current quiz and it is still an option to have the questions in a category, just not emphasized so strongly. I have paper prototype tested the UI with two elder female teachers from our university so far (with stunning success smile ), and more to come. After finishing with the testing I will be posting the proposal linked to http://docs.moodle.org/en/Developer_notes as Tim suggested above.

Now, what I would need is any documentation of the quiz module there is. What I'm looking for most is a requirements document of some kind: what are the things that quiz will enable users to do/get. This is not exactly the same as a feature listing; I also would need use cases or usage scenarios, on the base of which one could design to all the audiences that Moodle has.

Also, any technical documentation will come in handy. I have to design this primarily for our electronic exam service, but it is an important goal to make it work with moodle core or to be part of core in the future. I will search the development wiki, but will appreciate any redundant links as well. I'm still at Moodle 1.6 but as I've seen the new features I'm thinking about upgrading to 1.8 - the new UI will, right away, have to be compatible with future versions
of Moodle, anyway. If the interfaces have not changed that much and you think we could do the development work on Moodle 1.6, I'd appreciate if there were any notes on what to take into account. I understand that formslib/QuickForm will have to be used - is it possible to use this, for instance, also from 1.6?

My deadline for finishing at least a simple version of the UI (which will at least enable easy making of a quiz) is at the beginning of August 2007.
Average of ratings: -
In reply to Olli Savolainen

Re: Quiz UI development

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Now, what I would need is any documentation of the quiz module there is. What I'm looking for most is a requirements document of some kind

LOL, we should be so lucky.

The developer documentation that exists is here: http://docs.moodle.org/en/Quiz_developer_docs

Anything not clear from there, the only description of how it should work is to look at the code to find out how it currently works.

And if you you ever try to remove a feature in the name of simplicity, people here will tell you why you cannot possibly do that wink
Average of ratings: -
In reply to Tim Hunt

Re: Quiz UI development

by Olli Savolainen -
Yeah, I kind of expected that. Anyway, had to ask! smile I'll post the first suggestions hopefully within two weeks.
Average of ratings: -
In reply to Olli Savolainen

Re: Quiz UI development

by Olli Savolainen -
Still doing major paper prototype testing this week, so will see if I'll manage to present the prototype here this week.
Average of ratings: -
In reply to Olli Savolainen

This forum post has been removed

The content of this forum post has been removed and can no longer be accessed.
In reply to Deleted user

Re: Quiz UI development

by Olli Savolainen -
I currently have no information about plans to use calculated q's, though such plans may well arise as we go on with our tests.

I am interested to hear about any upgrades to moodle 1.6, as we are still using it. Also I'd like to know about any challenges downgrading 1.8 code to 1.6 (and perhaps solutions) since I will probably face them myself, too.
Average of ratings: -
In reply to Olli Savolainen

This forum post has been removed

The content of this forum post has been removed and can no longer be accessed.
In reply to Olli Savolainen

Re: Quiz UI development

by John Isner -
The "quiz being made" is the wrong emphasis, in my opinion. The question database should be the focus, and categories should be emphasized, rather than de-emphasized. A high-quality, well-organized database can be used to create quizzes quickly and easily. Isn't that the ultimate goal anyway? Emphasizing the "quiz being made" will not serendipitously result in question reuse, or sharability. Other features that support the creation of high-quality question databases should be added to (or preserved in) the current UI: category preview (facilitates question selection), quiz-question cross-reference (MDL-8974), and display of additional question descriptors (e.g., level of difficulty).
Average of ratings: -
In reply to John Isner

Re: Quiz UI development

by Olli Savolainen -
I do agree. However, these are more like two different usage scenarios: For our application, quizes are smaller units, with a relatively small set of questions. Alas, keeping things organized (more than is happens naturally via creating categories for random questions) is not a top priority. Moodle is an application in such wide use that even though it's good to encourage a disciplined usage style, it isn't up to us to decide which way is the most appropriate for all users at the end of the day. They must have the choice.

Organizing questions seems nearly obligatory in the current quiz creating screen, which gives functionality for (at least) two seemingly separate tasks: creating the quiz and organizing the questions. The screen, being crammed so full of functionality, is something that confuses users with minimal experience, and for our use it is to a disastrous degree (refusing to use the whole thing at all).

That is: I am not planning to de-emphasize categories, but as you also seem to imply, creating and organizing questions is a separate task from actually creating the quiz. I plan to enable users to create quizes without having to explicitly understand too much about categories, but helping them along the way to start understanding them.

Your feature requests indeed seem very relevant and I'm considering incorporating the first two of them to my UI prototype. The third one requires data structure changes so that's something that would perhaps require further discussion. Could you further describe what you mean by "category preview", please? smile
Average of ratings: -
In reply to Olli Savolainen

Re: Quiz UI development

by John Isner -
I don't find the current Editing quiz UI confusing. When I teach the quiz module, I explain that the screen has two halves. The right half is for the question database, and the left half is for the quiz being constructed. You have to put a questions into the database before you can add it to a quiz. Hence the two step process: (1) add question to database (2) add question to quiz.

In my opinion, the Editing quiz page layout in 1.8 is nearly perfect. There are two columns clearly labeled Question bank and Questions in this quiz. This is a huge improvement over previous versions. My only change would be to swap the left and right and halves of the page. People think from left to right (at least it is so in the Northern Hemisphere smile), so this would more closely model the two-step process (I would make the same suggestion for the Group and role assignment pages).

Organizing questions seems nearly obligatory

By default, all questions go into Default, so a user who doesn't care if all questions go into one big category can ignore the category menu. Hiding the category-related features under "Show Advanced" would cater to users with minimal experience, but I personally find any such "dumbing down" of the UI to be distasteful.

Thanks to Tim Hunt, "Category preview" is already a feature in 1.8. It is controlled by the checkbox "Show question text in the question list" (see screenshot).


Attachment show_question_text_checkbox.png
Average of ratings: -
In reply to John Isner

Re: Quiz UI development

by Olli Savolainen -
Hi,

I don't find the current Editing quiz UI confusing. When I teach the quiz module, I explain that the screen has two halves.

That's basically the problem. People are masters at adapting/adjusting; once you learn it by heart you can do it. But it's the technology that needs to adopt, not the users. Creating an exam/a quiz is in itself such a simple task that it shouldn't need to be taught to anyone, especially no one who has already gone through pedagogical education. The current quiz module does indeed do its job - but it doesn't sufficiently communicate to the user how the job, which the user wants done, is done. As we don't have and don't want to spend resources to educate teachers in using a pen and a paper, I don't think we should have to teach professionals how to create exams.

Granted, some of the concepts Quiz uses (such as a random question) are computer-aided exam -specific, and thus need to be taught. But things such as "how do I add a question to an exam" should be expressed to the user far more obviously than what the current UI does. This is a big challenge, and I won't solve everything, but I do believe that making it much clearer is possible. It ain't dumbing down, it's smartening up when things are communicated consistently smile.

I'm sorry to keep talking about the new UI and not showing it yet. Just want to make sure (as sure as my resources grant me to be) it actually is usable first.
Average of ratings: -
In reply to Olli Savolainen

Re: Quiz UI development

by Don Hinkelman -
Picture of Particularly helpful Moodlers Picture of Plugin developers
In general (without seeing two different interfaces side-by-side), I have to agree with Olli. I have a taught about ten workshops on Moodle to beginning teachers, and the quiz module is one of the hardest and longest to explain (versions 1.2--1.7). It takes minutes to get teachers building forums, chats, feedbacks, journals, assignments and choices. The quiz takes hours. I still can't explain all the problems with categories and attached media files. We are always have issues with that. I am warmly awaiting the new question bank in 1.9. big grin

Also we are heavy believers in student-generated content and student-created quizzes. So the interface needs to be *really* intuitive!
Average of ratings: -
In reply to Olli Savolainen

Re: Quiz UI development

by John Isner -
I don't think we should have to teach professionals how to create exams.

Many teachers who are new to Moodle have prior experience using computer resources provided by textbook publishers, including tools for making tests and quizzes like ExamView and MyMathLab. These tools are at least as complex as the Moodle quiz module, and they DO have a learning curve, even for professional pedagogues. But some good documentation and a few short videos are all that's needed to get teachers using these tools. Why should it be any different with Moodle?
Average of ratings: -
In reply to Olli Savolainen

Re: Quiz UI development

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Moodle 1.8 is close to release. There are no dramatic quiz-related new features. However, the main change in 1.8 was lots of work to make Moodle more accessible. As part of this, all the editing forms were converted from using lots of hand-written HTML, to using a PHP library: http://docs.moodle.org/en/Development:lib/formslib.php. This makes most of the forms in Moodle much more consistent and a bit nicer to use. This includes all the question creation forms, and the quiz settings form. Unfortunately, the page where you add questions to a quiz has not been touched.

The main people doing development on the quiz at the moment are here at the OU. We have two of us working on it pretty full time (me and Mahmoud).

I have been working on integrating Moodle with another OU system, in a way that is basically a replacement for RQP. Mahmoud has been working on improving the quiz navigation - screenshot below. This is now live to our students. However, while it works for our quizzes, it does not work with all possible quiz settings. Also the code is not yet cleaned up enough to commit to core. We hope to get this into Moodle 1.9.

We are in the middle of deciding what are our priorities for the next 6 months of development. We have a huge list of features we want to do, that by my estimates is several person-years of work, so we can't do everything. The things that are likely to happen are:

1. Make the quiz navigation work with a wider range of quiz settings and get it in to core. Hopefully for Moodle 1.9.

2. Make Adaptive mode quizzes work much better.

3. A couple of new question types. (Multiple choice with the scoring rules we like, and some variants on drag and drop.)

4. A workflow system, for when you want a formal process for creating and approving a quiz. This won't go into core. It will probably be a 3rd party module in contrib.

5. We don't have time to implement it ourselves, but we will probably foot the bill for the gradebook development work that Martin Dougiamas want to do. In particular, we want the import/export API.

6. I also have to spend some time developing another OU system related to eAssessment (to talk to the new grade import/export API).

7. Start building up a test suite, to try to make the quiz more reliable.

So, as you can see, the making the quiz editing page much nicer to use does not make it into the next 6 months on our priority list, so it would be really great of someone else could work on it.

Attachment nav.png
Average of ratings: -
In reply to Tim Hunt

This forum post has been removed

The content of this forum post has been removed and can no longer be accessed.
In reply to Tim Hunt

Re: Quiz UI development

by Joseph Rézeau -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Tim,

I am in the process of adapting my REGEXP question type for Moodle 1.8 and have just discovered the new formslib PHP library and am building from the question_edit_form class. This is a real improvement, although it took me some time to figure out which elements did what and what parameters could or should be passed.

I especially liked the validation($data) function, which performs all the necessary validation before the question being authored/edited is actually saved, refusing to save it if not validated, and precisely displaying the errors exactly where they occur. I could even manage to test that parentheses and square brackets are correctly matched in the regular expressions used in the Answer fields before saving the question, rather than afterwards as I had to do before.

Thanks to all who made this possible. And I'm looking forward to a more user-friendly, less cluttered Quiz interface later on, as described in your post.

Joseph

PS.- Could you please have a look at bug MDL-8576 which is really blocking me at the moment? (after you have recovered from the jet lag...)

Thanks in advance.

Average of ratings: -
In reply to Joseph Rézeau

Re: Quiz UI development

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
I'll do my best, but my jet-lag is actually worse today than it was yesterday. I don't understand that.
Average of ratings: -
In reply to Olli Savolainen

Re: Quiz UI development

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Clicking the 'end test' link on the screenshot above beings you to this summary page:
Attachment summary.png
Average of ratings: -
In reply to Tim Hunt

Re: Quiz UI development

by udara weerakoon -
i wrote a simple javascript to highlight the answered questions only for questions with checkboxes. only support 16 questions using cookies, and work at firefox 2. I'm not a professional programmer of php so i'm learning how we can handle such things using php coding
Attachment quiz_UI.jpg
Average of ratings: -