Sorry for the delay in answering. Actually, my resources are rather spare as well, I'm working part time 12h/week until summer, which will hopefully be full time and I'll have time to actually dig into things. Anyway, I was excited to get such supportive replies from you all. Thank you!
Don: as you can see from what's below, it will still need to be verified what's really critical, and of course if you already know about something that's problematic with the quiz UI (and this possibly verified through testing), that would be great. Once we get some prototypes, we'll be grateful for any sort of testing done on those, for example
paper prototyping. What do you think?
Tim: thank you for letting me know the status of your work. Hope we'll be able to help you with your workload
.
The main problems reported with quiz and new teachers by our support team have been as follows. However, we will still need further testing during the spring to actually know what are the most critical problems. Just to try to keep you up to date about what's going on here, I'll give you some of the thoughts we have so far. Hope I'm not being too aggressive with the following!
Question categories
The concept "question categories" versus the questions which are actually already in the quiz. Since we here are developing an exam system which is supposed to partially replace conventional paper exams, the natural mindset for teachers is: to create an exam you design questions and write them into the exam.
In contrast, moodle's current model requires the teacher to first
create the questions into a
category and then
add them to the exam, which has been reported to be very counterintuitive. They just want to put the questions directly into the exam (
Nielsen's heuristics:
Flexibility and efficiency of use); the fact that they are at the same time creating them in a default category could parhaps be hidden in an "advanced" screen. The system of concepts that needs to be understood isn't really communicated to users who don't already know how it works.
Even if they could grasp the idea, there's simply too much going on in the quiz building screen (including the subtabs). Most teachers don't want to bother themselves with sharing questions; however, many of the exams we will have
will have multiple essay questions randomized from categories (which is
experimental functionality it seems). This is something most teachers will want, but with the current UI, have no idea of how to create. We have way too many (new) teachers to teach all of them to use the current UI for that.
I'm thinking about initially supporting only the simplest of operations (see
progressive disclosure), and then having much of the current things behind an "advanced" button. Anyway, this is just the first thought, completely untested. Since managing questions and categories already has a separate screen of its own, it would seem simpler to keep those tasks away from the quiz UI, but this is problematic of course since the tasks really are related.
Tabs
The current hierarchical tabs in quiz management are rather confusing. If you have no questions you will be thrown to the edit tab no matter what, but there's no feedback about why that happens even though all the other tabs are active (links), too.
The active tab isn't really highlighted so it's hard for users to even know where they are; furthermore, the whole idea of "hierarchical tabs" seems problematic, and would probably be best replaced with a hierarchical menu on the left side of the screen (see for example winamp's or eclipse's config windows). Or perhaps the fact that there is a hierarchy could be made more visible (see, for example, firefox 1.5's - not sure about ff2 - tools -> options window). I'm not even sure that a hierarchical menu structure is
really required at all for quiz, but naturally redesigning the entire UI structure is something that will take more time than just the half hour I have right now to try to continue the discussion.
That is my 5 cents for now. Looking forward to hearing your thoughts about all this.
