Manual grading pop-up

Manual grading pop-up

by Tim Hunt -
Number of replies: 15
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Over at MDL-53603, some folks have raised what they see as a usability issue, but I am not sure. I was hoping that if I posted here, we could get some comments from some users.

The scenario is, you have a quiz set to shuffle the order of questions.

On the review page, for different students, each question has a certain number (in the screen-grab below Question 1). However, in the quiz reports, the questions are numbered according to the order on the edit quiz page (in this case Q4).

The difference is most noticable when you click the question score, to open that one question in a pop-up window. There, the link in column Q4 goes to a pop-up saying Quetsion 1. Apparently some teachers find this confusing.

However, that same pop-up can be reached from the quiz review page, by clicking Make comment or override mark.

Screen grabs of the problem

Is this really a problem? if so, what should we do about it?

Average of ratings: -
In reply to Tim Hunt

Re: Manual grading pop-up

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

I had never noticed that!

Interesting problem.

On the quiz report, it seems that it makes sense that a teacher would want to see the overall results for a question.  However, this is somewhat "administrative."

Maybe the quiz report can show results by student, but there should be an administrative "switch" to see it by question.  Hmmm, this could get pretty complex, but maybe worth thinking about.

Average of ratings: -
In reply to Tim Hunt

Re: Manual grading pop-up

by Daniel Thies -
Picture of Core developers Picture of Plugin developers Picture of Testers

I have used this and definitely consider the current ordering on the attempts report a feature not a bug. Shuffling implies that each question is independent of the others in the same attempt (order does not matter). If I have essay questions, I would prefer to grade all attempts of a particular question at the same time before going on to a new question. For automatically graded questions it also makes it easy to spot questions which are causing problems for students. If the teacher wants to grade in the order presented, selecting the attempt and using 'override or make comment' works for that.

It might help to have both slot and the shuffle number shown in the popup, but I do not know why. If might also work to allow teachers to toggle the ordering on the report, but I would not do away with the current ordering.



Average of ratings: -
In reply to Daniel Thies

Re: Manual grading pop-up

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I had a thought about this.

Perhaps it would help the teacher to see the original question number (in addition to what the student saw) everywhere. (E.g. on the review page too.)

We could do this by changing the display for any user with mod/quiz:viewreports capability. In the grey 'info' box we would display something like

Question 1

(was 4 before shuffling)
Average of ratings:Useful (2)
In reply to Tim Hunt

Re: Manual grading pop-up

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

I think that this is the better approach.  The quiz report needs to have all the questions in the same location but I think there is value in knowing what order the student took the questions in as well.  Not sure if you should have it the way you have or reversed - Question 4 (shuffled to position 1) or something similar.

Average of ratings:Useful (1)
In reply to Tim Hunt

Re: Manual grading pop-up

by Abhilash Mhaisne -

Hi Tim! Thanks for creating the post and images.

I think the order of questions in the report should be the same as original non shuffled order. So when the pop up window appears it should display the question number same as it was before shuffling, not at what number it appeared to a student.

Shuffled questions are for students! I would like to ask everyone here, why display the shuffled number of the question? 

Thanks and Regards,

Abhilash

Average of ratings: -
In reply to Abhilash Mhaisne

Re: Manual grading pop-up

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

Your post made me think about "Why and how do I use this report?"

My most common use is when a student writes me saying "I don't know why I missed questions 4."  I go to this report, see that the student got question 4 wrong, then I click on it to see what the actual question was.

If we show the unshuffled order, how will I know what question 4 was?

So I wonder if the unshuffled order report should be part of statistics reports for the quiz?

Or maybe the "spreadsheet" view should show the unshuffled order, then to see "Question 4" one should click on "Review attempt."  Or maybe there should be one more button (next to, or under "Download") that says "Show unshuffled."

Average of ratings:Useful (1)
In reply to Rick Jerz

Re: Manual grading pop-up

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

In that scenario, I think that you have to reivew the student's attempt, to see the question numbers as the student saw them. That is one extra click from the quiz reports -> Review page -> Q4. I don't think there is a feasible way to shorten that.

Average of ratings: -
In reply to Tim Hunt

Re: Manual grading pop-up

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

Yep, I can live with this.  So are we approaching agreement that the numbers on this report reflect the "unshuffled" order?  If so, do you think this change will be easy to implement?  Do you want this discussion to go on for a little longer, just to see if anyone finds problems with this new approach?  Does the word "Unshuffled" need to show somewhere on this report?

Average of ratings: -
In reply to Rick Jerz

Re: Manual grading pop-up

by Colin Rice -
Hi,

Taking feedback from within my own organisation, it is simply confusing to click on the question in the report and get a different number in the popup box. So I am in the camp that suggests always display unshuffled questions in the report and subsequent popups; shuffled questions are a presentation format for students only, in my honest opinion.


Cheers

Colin

Average of ratings: -
In reply to Colin Rice

Re: Manual grading pop-up

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

Your post makes me wonder if this gets even more confusing when the shuffled, or unshuffled questions are also randomly picked?

Average of ratings: -
In reply to Tim Hunt

Re: Manual grading pop-up

by Daniel Thies -
Picture of Core developers Picture of Plugin developers Picture of Testers

Okay in this case you may indeed want to know where question 4 was in the unshuffled order. For example that Q4 for this student is Q10 of the unshuffled questions so you can check on the attempts or statistics report how the rest of the class performed on the same question.

I like the idea of having both visible to the teacher on the popup.

Average of ratings: -
In reply to Daniel Thies

Re: Manual grading pop-up

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

My suggestion was to display this to the teacher everywhere the question is displayed. Not to have different displays in different places.

Average of ratings: -
In reply to Tim Hunt

Re: Manual grading pop-up

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

I might be just a little confused and apologize for asking for clarification.

Right now, the question (like #4) is shown on the report and in the "review attempt" of the quiz always as #4, which is #4's "shuffled" position.  You want to make both of these reports use the "unshuffled" position.  Is this what you are saying? One way, or the other way, but not both and not a mixture?

Average of ratings: -
In reply to Rick Jerz

Re: Manual grading pop-up

by Daniel Thies -
Picture of Core developers Picture of Plugin developers Picture of Testers

In Tim's example above. Clicking Q1 on the review page and Q4 on the attempts report both link to exactly the same question. If one numbering system is chosen for the display, one of these links will appear to be wrong. One solution is to display both numbers for each question. Another possibility is changing the order of one of the reports so the teacher only sees one set of numbers. However, that will create more difficulty comparing responses from different students or attempts.

Average of ratings: -
In reply to Abhilash Mhaisne

Re: Manual grading pop-up

by Abhilash Mhaisne -

Thank you for responding! Reading the discussion here, I can think of  another solution similar to what Tim mentioned :

In the popup window, display the unshuffled question number in front of question number. Add another string below it, say, "Appeared to student on number 4".

So 2 strings (Eg. original question 10, it appeared at no. 4 to a student) :

Question number 10

Appeared to student on number 4

What say?

Also, I have submitted the patch for unshuffled question number to appear on the bug page. But I think I'll wait for a final behaviour which everyone agrees upon.

Average of ratings: -