I am Olli Savolainen and I am working on the Quiz user interface this Summer. I have been studying the usability aspects during this spring already, and what we have learned so far about the users is here: Quiz UI redesign scenarios index
It is a set of six fictional characters (personas) who are described doing different tasks (scenarios) with exams.
Now you can help. It does not matter if you have not used Moodle Quiz or other electronic exams before. Please reply to this message, telling us how, and for what purpose, YOU are creating exams - electronic or not. Instructions below.
(The video contains roughly the same information as what is below.)
The way you can help is as follows:
- In the personas page, find a persona that matches your way of making exams (electronic or not).
- Read about how that persona is doing exams on the two other pages about making exams: Quiz UI redesign scenarios index
- Scan for differences between your way of working with exams, and the way your persona does it
- Tell us, in this thread, about what you found: How do you do it?
The goal is to get a new user interface (UI) in Moodle 2.0. Please also post this message whereever you think we could find teachers interested in exams, who might like to help. (However, please do bear the netiquette in mind.) Thanks.
(I would crosspost this to the following forums, too, since the users of the current quiz probably are best represented in the existing UI already - the most important is to hear the users who, for a reason or another, are not using quiz yet. I will wait if the video above to get properly embedded, though.
Thanks for working on this. I am probably closet to Ida, although my use of formal assessments differs in significant ways. Here I give an edited version of her's to describe how I use the Quiz module and and formal test type assessments in general.
Gary (Math Teacher)
Gary teaches several math courses in high school including Advanced Algebra, Calculus II, Statistics, Math Modeling, and an elective course in Software Development. He also works with his faculty and students in customizing Moodle to work well in the secondary school environment. He is quite busy, as are most high school teachers, and needs to economize on his schedule so that he can spend more time collaborating with his colleagues and students.
Quizzes, tests and exams: why and what
Such assessments are about:
- Evaluating learning of basic factual knowledge of the field
- Multiple choice question, short answers, and essay questions are used.
- Paper quizzes are also given when step-by-step processes must be evaluated.
- Automatically graded questions are especially beneficial when students immediately get feedback on their answers and have a chance to give it a second or third try. It also significantly reduces the teacher load while giving better feedback on the progress of individuals and the whole class on topics covered.
- Questions banks are generated from Examview software that is aligned to our textbooks and imported when possible to facilitate construction of appropriate questions. This also allows easy generation of sample tests for the students to study from.
Gary sees such assessments as beneficial since they provide measurable grounds on which to compare and grade learning of certain elements of instruction and for quick feedback to students. He wants to have his exam as automated as possible except when hand grading gives him better insight on student progress (seldom does he find that hand graded work does this when dealing with many students). He also believes that there are other valid assessments of student progress that must be included in evaluations including formative responses, measurements of process (rather than just outcomes), projects, and contributions to the development of the whole group of learners.
For Preparing questions, I have them generated by Examview whenever possible. I will then sometimes add an essay question or two that I decide on manually to check for understanding of something emphasized in class and that is generally more open-ended hence giving students a chance to be more creative.
The organization is therefore added by the external program which allows me to select from items taught in the textbook.
I also agree with Joseph's comment that the quiz module should go beyond just assessment and allow for quick feedback for students in checking their understanding and mastery of the material.
Sorry for the delay. I already took a bit of your text into Ida's persona . To clarify a bit more, I have a couple more short questions for you.
I understand that you use Examview for storing and organizing your questions, but Moodle for actually implementing them into an actual test/quiz/exam? Or is Examview doing that, too?
I downloaded Examview but since I mostly work on Ubuntu myself, I have not yet tried it out. What does it mean, that you generate questions in Examview? What do you get from Examview, entire question banks for importing to Moodle, or single question categories, or...?
Just to make sure: hand grading here equals essay questions, and automatic grading short answers and multiple choices?
Why are paper quizzes best for this purpose?
Could you further elaborate on what you mean by evaluating step-by-step processes, please?
Thanks for your efforts at re-designing the Moodle Quiz activity. I cannot answer your "persona" questions because I never use Quiz for testing, only for learning. Which leads me to the remark that - from my point of view - your approach is flawed by the fact that the only use you consider for Quiz is "exam". Could you please re-consider your approach to include uses of Quiz for learning or self-testing, something quite different from the "testing/exam" approach?
There is a question list template that I used to gather the questions, and I will publish them shortly. If you wish, after I publish those more open-ended questions, you can answer them instead, they do not make such assumptions as we have in the current personas. Do you think you might have time to answer such questions?
Or, if you prefer, we can try a Skype call and I will interview you with my moderate English and my horrible French ;).
Hopefully he'll check it.
I thought your description was indeed useful and adds depth to our understanding to this persona that might be the most important one in the long run. If you wish to make it more easily readable, below are my thoughts.
What I liked about the original text was that it was brief. With the other descriptions it is relatively easy (though this could be further improved) to see an overview of what kinds of different users are involved. So some of these might be useful:
- Add a brief summary in addition to the body text, which lists the main points
- Add bold to key concepts so that your writing is more easily scannable
- If you see appropriate, divide your description, moving some of your text into the two other pages with more practical, action oriented emphasis, to help programmers and UI designers find what is relevant to them at a given moment.
Ugh , back to the drawing board.
(I'm also wondering if the division onto the different pages could be made clearer, but maybe i was just being "dense" )
Feel free to move anything.
I have now published the interview questions. If you can, please answer to them in this thread, and please also tell us about anything that those questions may miss about your way of working, so we can get your persona included in the work. Also the background information form's questions are available, you might want to answer them, too. Thank you so much already!
Good Morning Olli,
I am just now trying to learn how to create assessments that the computer can score for me. I think the Ida persona is closest to what I am trying to do. I teach Chemistry, Biology and Earth Science in a charter school( Blended Learning Community). I see my high school students in a college -like schedule, so I desire to have activities for them online. I want Quizzes that can show them and me that they have understood and mastered the homework assigned from the text.
I have just discovered SCORM and Hot Potatoes. I attended CUE this spring. Our Moodle Administrator last year did not include SCORM. I am asking for the module to be included in the update.With 125 students, I am overloaded trying to grade traditional homework papers and tests. I have tried to use the Quiz creator within Moodle 1.8 once and got frustrated by the poor word processing within it. Mistakes I deleted persisted; one question said it was right and wrong at the same time...then I was out of learning time.
What I want is a bank of premade Quizzes and tests I can select from that the computer will grade and just give me results. I don't have time to write all these assessments for three different courses( Earth Science, Biology and Chemistry). I am using HOLT Chemistry and Prentice-Hall Earth Science - both have question banks on Examview. Prentice-Hall Biology( Miller and Levine) does not. The authors' website has nice Chapter Self-tests, but no way to collect data on how the student performs out there.
I hope my frustration and inexperience benefits this project! Thank you for working on this stuff! Shannon Parrish, Horizon Charter School, Roseville, California