I am not completely unsympathetic to this, and during the Hackfest at the Edinburgh MoodleMoot, I grabbed Stuart Lamour and Bas Brands, who often go on about usability issues in the forums, and we went through the quiz UI from the student point of view, to see what could be done. Wow! that was 2 months ago now. Since then, I have had this scrappy bit of paper on my desk, and been meaning to post it. Here is the answer:
Grrr! I did mean to explain that diagram. (What, you mean it is not perfectly clear ) but instead I just discovered MDL-46069 after typing for about 20 minutes. I will explain at some point.
OK, short version:
Note, the reason I am posting is so that you can comment about whether these are good ideas. Please do.
- At the start of the quiz, only show the "Are you sure you want to start this quiz now" if the quiz has a time limit. At the moment, it is shown if there is a time-limit or a limited number of attempts, but there is only possible harm in starting an attempt if there is a time-limit.
- If the quiz has a password, this is currently shown after the Are-you-sure. (Though other checks like https://moodle.org/plugins/view.php?plugin=quizaccess_honestycheck are shown on the same page as the password, if both are enable.) Merge the time-limit Are-you sure onto this page, so there is never more than one page to click through.
- At the OU, we have some nicer styling of the quiz navigation panel. Get that in to Moodle core.
(Note, I promised to do that once before, several years ago, but it never happened.)
- For now, at least, we are going to leave the Summary page, and the 'Are you sure' when you click "Submit all and finish".
- After that, when (if) the student gets to the review page. We should change to to show all the questions on one page by default. (Perhaps only if the quiz has fewer than 30 questions, or some limit like that?)
- Instead of the 'Finish review' link at the end of the review, we should instead put 'Back to course' and Re-attempt quiz' buttons right there. There would be no direct link back to the quiz information page.
- Also, we should always show a 'Back to course' button on the quiz information page.
- I wonder if it would be better to move the quiz 'Overall feedback' from the top of the review page to the bottom?
If you are wondering what I think, I am pretty sure 1.-5. are all good ideas. I was not sure about the others initially, but the more I have thought about it over the last two months, the more sensible they have seemed, but I am still not sure.
I'm sorry it took me so long to post these. I would really like to know what you think. If we do this, it would be nice to do it in time for Moodle 2.8 in November. UI changes like this would only go into a major release like that.
I'm good for 1 - 6 (6 will cure an annoyance more than a usability issue).
9 I'm less certain about, but I think that's because I have become accustomed to the overall feedback at the top. Moved to the bottom, it becomes a summary after I've reviewed all my answers, so that makes narrative sense.
Thanks for the warning about Firefox and atto.
7. In the table at the top of the review page, there are links to review your other attempts at this quiz (if you have other attempts). Probably another bit of UI, like the breadcrumbs, that is too subtle for anyone to discover or use.
9. Another option would be, repeat that whole review info table at the bottom of the page if the quiz is long enough (> 10 questions?)
Actually, I suppose a better way to think about it is this: If you are a teacher grading a paper test, do you write the big fat F and your overall comment at the top of the first page, or at the end?
The links to other attempts are only present in the teacher view. Students don't see that little bit.
I write the big fat F in the top right corner of the page and do overall comments immediately underneath.
not going to argue the toss with you on that for sure. I do want to outline my reasons for suggesting feedback at top....grade at the bottom however....reasons follow:
1. have spoke to many students over the years...both undergrad and post-grad and been both of those type of student myself....both ftf and online learning envs.
2. I conclude from that, when the feedback is at the top of the scroll and the grade at the bottom...there is more chance of the feedback at the top-being read..........the words chosen by the teacher can often signal what is to come...as a student...and having talked with students-they know this.
3. When both are at the top...same section.....more often than not........I could be wrong.......it is more of a quick scan at the grade and off to the pub!
Yet, the F score...is the one person who needs to really....read that feedack to avoid a trail of such marks/grades and avoid the inevitable.............do those feedback sections get read.....probably...but the best way is with guidance with the tutor.....and on...and on.......
Makes sense to me. With that said, the overall feedback option isn't really a detailed set of feedback that we can guide students with. What you're suggesting really makes sense though on paper assignments where rich feedback is given.
oh my, this- I think is a whole new thread...not in quiz forum though eh.
When you say: feedback option isn't really a detailed set of feedback that we can guide students with' I struggle with that idea, to be honest, but I shall shut up about it for now ;)
Paper feedback-eh? Does that still happen then! Only joshing with you Joshua-I understand your point.
Shame that there still does exist the mindset, which includes a demarcation between online/electronic feedback and paper feedback....(the latter perceived as affording more depth) I disagree with that perception, although over time I suspect practice will change markedly and the balance will be more evident, perhaps.
And, I still stand by my view, in that the grade/mark at bottom of page might be better placed for learning purposes. I am glad the plan to keep written feedback at the top of scroll is a popular one.
Most of those match up with the feedback I've received from users. #6 and #9 I've not heard anything about or have any opinion. One of our local customization is adding "Back to course" on pages where it might be reasonable for a student to return to the course at that point. I fought that a bit, but others in my organization convinced me to do it, and now I think they were right.
The end of the quiz does seem to have a lot of clicks between answering the last question and being completely done, heading back to the course page. Anything to reduce that is good.
Eliminating "are you sure" in cases that don't have bad consequences is a great idea. We don't want to train users to ignore those messages by popping them up too often. Besides, I clicked "Start Attempt", so yes, I want to start an attempt.
I'm glad everyday objects aren't like Windows^H^H^H^H^H^H^H computers.
"Are you sure you want to unlock the car door?"
"Are you sure you want to open the door?"
"Are you sure you want to get in the car?"
"Are you sure you want to start the car?"
"Are you sure you want to put the car in reverse?"
"Are you sure you want to get on the highway?"
"Warning - you are late for work!"
1-5 and 7-8 are fine.
6 I'm less sure about. It certainly needs an extra refinement that if there are questions brought in through opaque then it should remain at one question per page for default (because we know that multiple OpenMark questions on a page can fail to render properly).
9 I'm not in favour of.
actually....very good point. I would be inclined, if it were me-to keep the feedback at the top (that is what needs to be read-and is often not!) and the grade at the bottom...in line with the same SOP.....that can often exist across assignment feedback templates for Ed per se (course, I am talking about my experience here in UK)...so might be a narrow view...or not
There seems to be a surprising amount of consensus about this. People are happy with all the proposed changes except 9, which almost everyone thinks is bad.
Actually, 9. is something is added to my forum post at the last minutes, not one of the changes that I have been thinking about for a while, so I am not so surprised it turned out to be a bad idea.
Because there seems to be consensus, I created MDL-46090 to track this work. (I also added a couple of related accessibility issues there, because they seemed relevant and I might fix them at the same time.)
To answer some of the questions raised:
6. Phil: issue with qtype_opaque not working with show all on one page. I don't think we can let this hold back improvements to the quiz. Instead, we need to fix qtype_opaque. (I wonder how hard that is?)
7. Joshua: difficultly of navigating to review of other attempts, in quizzes with multiple attempts and not immediate review. I am not sure how much of an issue this is. Maybe make inter-attempt links available to all users, not just teachers?
7. Dawn: getting rid of back to info link at the end of review is bad. Maybe, but on balance I would still rather remove it. Once I have coded up the change, we can see how it looks before the change is actually added to Moodle.
7. Tim Lowe: Only show Re-attempt quiz button if that is possible: of course.
8. Joshua: back to course link on quiz info page, does that go against the Moodle standard? Yes, that is my concern, so I am still unsure, but quite a lot of people have asked for it over the years.
I hope that is everything.
More thoughts on #7:
Perhaps the reason why the inter-attempt navigation has not been traditionally shown to students is that there are so many review options that it would be difficult to rationalize when to show them and when not to. Having a link to an attempt that students are not allowed to see would be very confusing. Having a list of attempts where somethings are not linked would be just as confusing.
One of the nice items on the quiz info page is that it shows when the review will become available to students.
Perhaps an alternate option would be to:
- If there has only been one attempt so far only give a return to course and reattempt quiz buttons.
- if there have been multiple attempts so far then also add in a "see my other attempts" button. That would take them back to the traditional quiz info page.
Well, I won't rush to implement this one. I will go on thinking about it.
I will just note that any situation where we show a user a link that just takes you to an error saying "You are not allowed to see this" is a serious bug. It should just not be done, and I would not do it here. It is clearly possible to display this properly, since we display the right links in the table on the quiz info page.
I had a lot of discussions with coworkers (course creators, mostly) and this list of changes, mostly 1-3 and 7-8, would make them very happy. 4-6 would be appreciated as well, but are not as critical. Good stuff!
I need to finely tune my brain for this Tim....back after some zzz......s.
have taken the liberty to order the points in terms of what I see as priorities for the user...now I hope there is laughter there....because if you really think about that...how could you have expected me to have done anything other than that!
Here you go..(and yes, that F needs to placed very gently across proceedings-agree)
1. At the OU, we have some nicer styling of the
quiz navigation panel. Get that in to Moodle core.
(Note, I promised to do that once before, several years ago, but it never happened.)
2. I wonder if it would be better to move the quiz 'Overall feedback' from the top of the review page to the bottom?
3.After that, when (if) the student gets to the review page. We should change to to show all the questions on one page by default. (Perhaps only if the quiz has fewer than 30 questions, or some limit like that?)
4. Also, we should always show a 'Back to course' button on the quiz information page.
5.Instead of the 'Finish review' link at the end of the review, we should instead put 'Back to course' and Re-attempt quiz' buttons right there. There would be no direct link back to the quiz information page. SHAME, I THINK THAT WOULD BE USEFUL.
The rest are all linked together/or involve no change-as far as I can see.
(Was eventally allowed to reply after first resetting my forgotten password and then realising that when I'm not I need to "enrole in the course" really means "join the community"! )
I'm happy with all of these, except 9.
In 7, I assume the "Re-attempt quiz" button would only be displayed if re-attempts were permitted!
Tim Lowe - where did you see it say "enrol in the course"?
for creation of complex form stuff - analysing the workflows of survey monkey and google forms could be a nice user friendly source of inspiration.