Simple Quiz building user interface

Simple Quiz building user interface

Olli Savolainen -
回帖数：28
During Summer 2008, I will be working on a simplified alternative UI for creating and managing quiz content. The basic idea of the new UI is similar to the proposition I presented to you all nine months ago (in the project I was back then, we lacked resources for the actual implementation). After having further discussed the issues with Tim Hunt, the approach is now somewhat different. At this point, I'm asking for your thoughts about the current proposition, below.

The idea is to provide an option for Moodle administrators to offer an alternative, simplified UI for especially the novice users to get to started quicker building their quizes (similar to progressive disclosure, they can then move on to the full UI later, if needed). Currently the UI is so complicated and miscommunicative that unexperienced teachers are refusing to use it (I am quite sure that this is not the case just in the University of Tampere, where I come from). Nonetheless, I believe that even experienced users will benefit from making the UI better communicate what and how it is meant to be used, and by showing more of the information required at each stage of the process of creating or managing a quiz.

The proposition includes two new screens, each organized into a tab of their own. (See also: the appropriate parts of the old proposition). As we go on, we will need to confirm everything conforms to the current conventions of Moodle, but I don't think will be an obstacle.

Diagram of the basic idea

The first screen shows the actual content of the quiz. Here, you can

  • add new questions,
  • add new random questions,
  • add new questions inside random questions,
  • manage the properties of an individual question in the quiz (mainly “maximum points for this question”),
  • use a question or a random question as a template for new ones (that is, make copies of questions and edit them),
  • set the maximum grade for an exam (like in the old UI)
  • remove questios from the quiz

The second screen is a tool for organizing the questions within and into pages, (and for adding new pages). This functionality is, using UI design terminology, already a mode inthe current UI (checkbox “Show the reordering tool”), but moving it to a tab of its own makes the mode more explicit and thus, helps the user to recognize it and to act accordingly.

To understand further UI design justifications for new screens, please see the new Quiz UI redesign prototype presumptions page in Docs. The main question that document answers is: Why can't we just make smaller fixes to the current UI?

After developing these new screens, we will have an UI to match the users’ natural conceptual model(s). This will be a good start on which to further develop the user experience. In the future, if feedback from the community (you!) is positive and further usability testing shows that the new UI works for the different user groups, this new UI may replace the current UI altogether. For that to happen, further features will need to be introduced, though, and further discussions had. There are further issues down the road, such as how to make the overall quiz navigation reflect the actual relationship between the question banks and quizes (this is also central to users who mostly just import their questions), but luckily we can move in small steps and just have a separate, simplified UI for just creating/managing quiz content, at first.

I will probably work on this funded by the Finnish equivalent of Google's summer of Code (this link seems to suffer downtime today) - or if not by them, then by another interested party. After funding has been confirmed, I will disclose the more detailed project plan.

平均分： -
回复Olli Savolainen

Re: Simple Quiz building user interface

Tim Hunt -
Core developers的头像 Documentation writers的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像
This is really just to confirm that Olli and I did discuss this a bit by email, and I think it is a worthwhile experiment. The important thing is to keep some continuity between the way Moodle currently works, and the new interface, and I really like the diagram in Olli's post, that is a really neat summary.

Note that Olli's proposal is entirely about user interface, all the behind-the-scenes stuff about how the quiz design is stored in the database does not change at all.

Our hope it to find a way for Olli to do this work so it is easy for people to download it and try it out. Then we can decide whether to use it in Moodle 2.0, but I hope we can.

I a surprised no one else has comment on this yet.
平均分： -
回复Tim Hunt

Re: Simple Quiz building user interface

Joseph Rézeau -
Core developers的头像 Particularly helpful Moodlers的头像 Plugin developers的头像 Testers的头像 Translators的头像
Tim "Our hope it to find a way for Olli to do this work so it is easy for people to download it and try it out."

I am willing to try it out as soon as it becomes available.

Joseph

平均分：Useful (1)
回复Tim Hunt

Re: Simple Quiz building user interface

Chris Potter -

I think it's a great idea to clean it up a little and it looks great so far. I'm surprised it hasn't caught on much sooner. One thing that seems to get me is that there are certain times when our instructors want to use the same quiz but want to empty out all the current questions and our current version doesn't allow us to do that (1.8). Maybe it's already fixed, but little things that can improve the way we build quizzes can certainly benefit faculty. Our faculty often make note of the amount of content on the screen and how it seems overwhelming. They really just need to take one section at a time, but I agree that it can be a lot to take in, especially for the less savvy of the users. I thought about an AJAX-type interface for "drag and drop" adding and removing of questions, but haven't gotten around to it (and it doesn't look to be in my immediate future).

Just thought I'd give my .01 cent. 微笑

平均分：Useful (1)
回复Tim Hunt

Re: Simple Quiz building user interface

Nadav Kavalerchik -
Core developers的头像 Plugin developers的头像 Testers的头像 Translators的头像
i'd love to try it out too 微笑

and please have a look inside the following links for
already available GREAT pieces of JS code for UI
(it's cross-browser code i use in some places)

jQuery: The Write Less, Do More, JavaScript Library : jquery.com

X Demos and Applications ::www.cross-browser.com/toys :: cross browser javascript code
DOJO - Ajax, events, packaging, CSS-based querying, animations, JSON, language utilities, and a lot more. All at 23K.
MochiKit is a highly documented and well tested, suite of JavaScript libraries that will help you get shit done, fast. We took all the good ideas we could find from our Python, Objective-C, etc. experience and adapted it to the crazy world of JavaScript.

script.aculo.us - script.aculo.us provides you with easy-to-use, cross-browser user interface JavaScript libraries to make

your web sites and web applications fly.[animation framework,drag and drop, Ajax controls DOM utilities, and unit testing.]

平均分： -
回复Nadav Kavalerchik

Re: Simple Quiz building user interface

Olli Savolainen -
Thanks Nadav, 微笑

It seems though that Moodle is committed to using Yahoo's library so primarily we will stick to that I guess. In any case, the basic premise of good AJAX development is that the foundations must first be built to work without the help of javascript, and AJAX-like functionality is an additional plus. There are indeed elements in the new UI that could benefit from AJAX, but at this phase I will first make sure the foundation is solid.
平均分： -
回复Tim Hunt

Re: Simple Quiz building user interface

Olli Savolainen -
Thank you all for your encouraging feedback. The finale for the current funding application is on March 28th, I will report back to you as soon as I know more.

Chris, I also think that being capable of emptying the attempts is an important use case, though it is beyond the scope of this UI project.

Drag and drop has its benefits, however often users don't realize they can use it, especially in the web. If it's accompanied by a more obvious method to do the same thing, it can be effective.
平均分： -
回复Olli Savolainen

Re: Simple Quiz building user interface

James Williamson -
Hi: we are interested in following this discussion too. Thanks, Olli, for taking on this important work in Moodle.
平均分： -
回复Tim Hunt

Re: Simple Quiz building user interface

Olli Savolainen -
Tim & others,

What was mentioned in Monday's developer meeting about the end of May deadline for specs didn't get very clear to me yet. I am currently working on a new version of the prototype (basically the goal of which is to simplify yet more without losing functionality) to have something based on which to interview teachers, but the spec based on which I will do the implementation will be ready late May. So if I have the functionality of the UI defined by then, as well as some mockups, will that keep doors open to possibly have it in Moodle 2.0?

The thing is, I am not sure if some of my ideas will require additional DB structures to keep things stored, even though no new features are being planned.

The main difference is perhaps creating categories on-the-fly in the background while creating a random question. Since the user seemingly proceeds in an order different from the current UI, there may be a need for a data structure to store the states of the UI (which do not as such affect the actual data structures of Quiz or Question Bank).

For example, hypothetically - though this perhaps is an extreme - the interviews might show the following: that one of the natural processes different teachers have would be to support first addding a generic question to the quiz, writing the question, and proceed to just writing all the questions first (without even specifying question type) in brainstorming mode, and only then continue to the "formalities" of providing feedback or setting the choices of a multiple choice. In this case, we would not want to add the questions to the actual quiz before they have gone through the "formalities", but would need an intermediate structure for the UI to store the incomplete questions. Of course, such a feature would need also full community support, but I guess you see my point.

As mentioned at least in the blog, the first three or so interviews will take place next Thursday in Haaga-Helia University of Applied Sciences, Helsinki, Finland.
平均分： -
回复Olli Savolainen

Re: Simple Quiz building user interface

Tim Hunt -
Core developers的头像 Documentation writers的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像
Olli, this is nothing for you to worry about. Moodle 2.0 is not going to be released any time this year, so anything you do and finish over the summer can certainly be included. Ditto for all the Google SoC things.

The reason I raised the issue of deadline in the developer meeting is that, unless you have been around in the community for a few years, you will not believe how long Moodle releases can drag on in an 'almost finished' state. There is an aweful lot of changes proposed for Moodle 2.0, and I think that unless Martin D starts imposing deadlines to try to get people focussed, the release is going to be a long way off. However, I can also guarantee that even if deadlines are imposed, there will also be exceptions made on a case by case basis.
平均分： -
回复Olli Savolainen

Re: Simple Quiz building user interface

Olli Savolainen -
I got accepted for funding in Kesäkoodi. Woo hoo! 微笑 Thank you all for your support, this is a good starting point.

I have set up a development reporting blog (the outlook of which will change shortly) though actual development will not get into full speed before June.
平均分： -
回复Olli Savolainen

Re: Simple Quiz building user interface

Olli Savolainen -

Came to think that the status of the Quiz should be readily available to users at any point. This is because a teacher may return to a quiz after a long time of not having used it, and in order to continue working from where s/he left off, it is crucial to know - quickly indeed - where it was (that s/he left off).

One point of inspiration was the Wordpress Dashboard (screenshot), which gives the status of various dynamic parts of the software. (The screenshot is from the admin panel of the project blog, that's why the main title of the page is what it is.)

Having a Quiz Dashboard for each quiz is an option, or:

I'm wondering if adding just a status line as a relatively prominent element everywhere in the UI would serve the user. The status line would, first of all, show the current principal status of the quiz, communicating things like (though probably not in this format):

  • "The quiz has no questions"

  • "The quiz has x random questions and y normal questions" (i don't know whether it's relevant here whether they are random or not)

  • "The quiz is visible to students"

  • "The quiz will open in 5 days for 3 days"

  • "The quiz is closed and has been taken by 23 students"

  • "The quiz has 4 ungraded attempts"

  • ...

Also, the status line would have links to all the relevant places to get more detailed information about the current state of the quiz (view student attempts, quiz statistics, ...).


平均分： -
回复Olli Savolainen

Moodle compliance feedback and competitive products details, please

Olli Savolainen -
Hi,

Most importantly, right now I welcome comments about what in the current proposal needs to be made compliant with the rest of Moodle. This is: which of the elements are not possible in moodle since they are too foreign to the users, and which elements could these be replaced with? Would it be possible to introduce new elements in the moodle element sets and themes if sufficient reasoning turns up? If I get sufficient feedback, I will draw another prototype/modify the old ones and possibly briefly even paper prototype test it.

Quiz editing main screen
Question organizing

Also, I am planning some sort of a competitive analysis, charting the features of some of the following: sakai, olat, ilias, atutor, and others. If you can tell me about the UIs of open- or closed-source learning platforms' Quiz creating UIs or perhaps even provide screen shots, I would be most grateful. I am most interested in how the workflow of creating a quiz differs in other systems from Moodle Quiz, but also how the actual UI is laid out is important.

I have started to do some preparatory work and gotten contacts with whom to discuss usage scenarios so that we are working on the right assumptions. I am gathering questions to ask from teachers and support people to gather the data. I will post those questions also here as soon as I get them to a more final state.

I am also thinking of using the Fluid Reorderer in the paging tab since seemingly moodle already has cooperation with them.
平均分： -
回复Olli Savolainen

Re: Simple Quiz building user interface

Olli Savolainen -
Just a brief status report.

Yesterday I had a brainstorming session with a trusted fellow HCI student, Ivar Ekman, with whom I usually get amazingly productive and who has lots of ideas himself. I have some changes to the prototype I am considering.

I am currently planning intervies and gathering the questions for interviews with Finnish teachers from several universities and polytechnics - this is in order to find out answers to the questions I have about the different features of the new UI, but to learn about the processes and profiles of teachers as thoroughly as I can.

I would definitely like some feedback, also to the above questions, even though I have already found ideas about how to make the new UI better 'fit in' to the Moodle UI. I can not do this without knowing what everybody here thinks, you know. 微笑

I am also keeping the project blog up-to-date as we go on.
平均分： -
回复Olli Savolainen

本讨论区帖子已移除

本讨论区帖子的内容已移除，无法再访问。
回复删除的用户

Re: Simple Quiz building user interface

Olli Savolainen -
Pierre,

I do not quite understand what you mean. The operations I mean to implement are in already present in the application, just the way they are presented to the user changes.

I can't parse the meaning of the language in your first bullet point in my head. Moving questions from category to another is possible already, and so is moving questions within a quiz?

Mais je l'apprécie, en tout cas! 微笑 Ehm, I guess you wouldn't really put it that way in French? (I am planning to go to France to actually learn the language next semester.)
平均分： -
回复Olli Savolainen

本讨论区帖子已移除

本讨论区帖子的内容已移除，无法再访问。
回复删除的用户

Re: Simple Quiz building user interface

Olli Savolainen -
Hi Pierre,

The main point of my work is to allow teachers to follow the process they would use anyway when making any quiz, off- or online. That is: to make the experience of creating a quiz as seamless as possible, so that teachers can concentrate on what they do best, the actual pedagogical work. The current (1.9) UI makes it impossible for many non-IT-oriented teachers to use Quiz, as mentioned in various documents. As such, "play[ing] with the interface so that it appears more convivial" could not be further from reality. See more info about the basic idea of usability: A Business Case for Usability

Add new questions inside random questions: This, in effect, means adding new questions to the category of the random question. In the following prototype, I plan to have it as follows:

[--Select question type--] [Add a new alternative to random question]

Do you think this would express the actual functionality better?

Use a question or a random question as a template for new ones: This is a bit of a tricky issue, I agree. I'm glad you asked.

Yes, the principal idea is to allow users to make further random questions from the same category. Thus the word "duplicate" is a bit misleading since it may implicate that the new copy is completely independent. (There is, however, also the use case of actually wanting to duplicate the category, too, but I don't see why anyone would want to do this since it potentially leads to duplicate questions in a student's quiz.)

The problematic design goals, partially in conflict with each other and left unaddressed by the current UI, are as follows.

Goal 1: Allow teachers to see the content of the quiz directly
Purpose:
Visibility of system status: It is hard to manipulate what you can not see
Error prevention: If you cannot see a question's content to verify its identity (which question it is), you may assume it is something else and for example, accidentally delete it.
Recognition rather than recall: "Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. [...]"
Match between system and the real world: When you create a quiz on pen&paper, it is possible for a teacher to have pretty much everything related to the quiz (registrations, questions, tools to edit the questions with) as physical or concrete objects - what they see is what they get. This is the real challenge here: as a random question IS in essence an abstraction, it is hard to present it visually: there is no longer such a thing as "this is exactly what the final quiz will look like".

In other words: if there are two random questions in a quiz, how should that be presented?
  1. The current approach: Close our eyes from the issue completely by showing nothing but the name of the random question. The teacher has to know that a random question is something that comes from the question bank, the idea and relationship to quizzes of which they need to understand. They need to understand that the text in the name of the random question (probably, no guarantee) comes from the name of the category, and then navigate to the question bank, and then navigate to the category in question. By this time they have often forgotten what they were supposed to do with the random question or which random question it was they wanted to manipulate.
  2. Show each instance of the random question in a similar manner:
    1. Make it visually clear that the question in fact is a random question and it contains several alternatives.
    Show 2. a small number of the alternatives 3. the amount of different types of questions in the random question and 4. link to the category to allow seeing and manipulating the rest of them.
    Two problems here: first of all, a novice user may not understand what a category is, and understanding that isn't required to create a random question. But if they do not understand that there is an abstract question bank behind the scenes of every random question, and if there are multiple random questions from the same category in the same quiz, the user may get confused when manipulating one random question causes the other random question to change, too. So the link to the abstract category may need to be shown - to some degree - after all.

    One possibility is to at least signal with every random question that "there are N random questions in this quiz from this question category".
Goal 2: Make the UI as simple as possible without sacrificing important functionality.
Purpose: "Aesthetic and minimalist design: Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility. "
Problem: visualizing the relationship between random questions and the question bank may clutter the UI, and thus this goal conflicts with the previous one.

I hope to have further discussion about these issues, since they certainly are not obvious and I am sure there are people here with expertise about, for example, which goal is perhaps more important.
平均分： -
回复Olli Savolainen

本讨论区帖子已移除

本讨论区帖子的内容已移除，无法再访问。
回复删除的用户

Re: Simple Quiz building user interface

Olli Savolainen -
Alright then. I guess I am a bit touchy about people seeing usability as just something like beautifying things on the surface. So, just to be clear, in which case did you think I am creating new problems, in your original sentence?

The question preview link is there in the random question, it seems that I've forgotten to include it in the old prorotype in the other question shown.

I do admit that I have a bias against documentation: it is a secondary means of helping the user understand what is going on. For me as a UI designer, it is important to focus on first trying to do everything I can to communicate the structures to the users by other means than documentation.

Nonetheless, having clean, compact documentation written from the standpoint of the user (here, teacher) and not that of the system is very important, and as the user does something more than just single quizzes with Quiz, they will need it. I do however still believe that teachers may not need that just for adding (single?) random questions into their quiz, as long as they have the concept that in an electronic exam, each question can be randomly selected from a predefined set.

平均分： -
回复Olli Savolainen

本讨论区帖子已移除

本讨论区帖子的内容已移除，无法再访问。
回复删除的用户

本讨论区帖子已移除

本讨论区帖子的内容已移除，无法再访问。
回复删除的用户

Re: Simple Quiz building user interface

Olli Savolainen -
Good, thank you, you gave me more things to take into account!

I agree, the user should be made aware of all the "unstabilities"/dangers of using random questions whenever appropriate. In any of those unstabilities, (except perhaps the third point (match)), it is very much possible and preferred to warn the user in the UI, when it is current to the user, and not in the documentation where the user won't find it at that time.

All of the actual questions do not have to be shown at once to the user, in order to verify that there are enough questions in the category: We can tell the user the random question/category has N questions, and that the quiz has M questions from that category. If M>N or even close, we will warn the user who might not understand the implications of M>N.

I maintain that even though a random question itself has lots of features, not all of them need to be stuck down the user's throat at once. Not all users need to have their questions in reusable categories even though that is the preferable thing to do - they just want the security provided by random questions (different students have different quizzes and thus cannot copy each others' answers so easily). They just want the quiz, and the category/question bank is extra. The category can show through, but we can not require the user to actually understand the relationships of the concepts thoroughly.

Below is an extract from my e-mail to Tim, when I was asking him to mentor me in this project, which he finally agreed to do. (hopefully I understand the concept of paradigm shift myself 微笑 )

"I wonder if taking this out of the software development world will help understanding the idea: To me, the situation seems similar to what a paradigm shift is about. People don't routinely switch the paradigms, which they think within, but have to work long and hard to understand a foreign paradigm before&while making the shift. Before they do that, it is not helpful to explain anything to them using the concepts of the foreign paradigm (in this case, meaning quiz's native conceptual model) since the foreign paradigm just seems wrong if you evaluate it from the standpoint of another paradigm (in this case equivalent to the conceptual model they've gotten used to while making pen&paper exams). This is exactly what is going on when novice users try to make a quiz with the current UI: being told that in order to make a random question you need to put questions to a category and then add that category as a random question to the quiz does not help, since in order to really understand the sentence in the first place you need to have fluently in the back of your mind the conceptual space these concepts are in, and the relationships between the concepts. "

And we don't need that. It is entirely possible to allow the user to advance at their own speed.

"Novice users see the quiz itself as the fundamental unit, since that is how exam making works in the real world. They need not bother with the idea that questions also exist in another dimension, the question bank. The best we can do for these users is allowing them to just create the quiz and ideally to give meaningful category names for their questions. They do not yet need to have a concrete understanding of the fact that the questions in the categories are somewhat separate entities from the ones in their quiz. Also, allowing them to create random questions directly into the quiz is feasible by creating the necessary category - transparently to the user - when creating the random question.

However, in Quiz, the eventual full model is that which also includes also categories etc., in addition to just the Quiz content. For use cases that involve exporting importing questions or keeping them organized, users will need to understand the full set of concepts. Still, we cannot shove the full set of concepts, foreign to the novices, down their throats by force. The solution is progressive disclosure (http://www.useit.com/alertbox/progressive-disclosure.html), which we can have to a degree even if the new interface is a separate module: users first use the UI that mostly speaks to them with only the concepts they already know (with the exception of random question) but manipulates the same data structures as actual Quiz. In this UI users may gain "conceptual handles" (such as a category), which they can use if they choose to learn the concepts required when using the more advanced UI."

That is: it seems to me that you are implying one and only one correct way to understand and use random questions and categories. But what I have learned from teachers who do not know Quiz is that their desired way to use the software is in contradiction with that "correct way". We can, and should support that.

平均分： -
回复Olli Savolainen

本讨论区帖子已移除

本讨论区帖子的内容已移除，无法再访问。
回复删除的用户

Re: Simple Quiz building user interface

Olli Savolainen -
Thank you again for all your feedback, Pierre.

Still working on my interview questions...

Olli
平均分： -
回复Olli Savolainen

Re: Simple Quiz building user interface

Nadav Kavalerchik -
Core developers的头像 Plugin developers的头像 Testers的头像 Translators的头像
what's the status of this beautiful quiz interface idea ?
平均分： -