Manual grading pop-up

Manual grading pop-up

von Tim Hunt -
Anzahl Antworten: 15
Nutzerbild von Core developers Nutzerbild von Documentation writers Nutzerbild von Particularly helpful Moodlers Nutzerbild von Peer reviewers Nutzerbild von Plugin developers

Over at MDL-53603, some folks have raised what they see as a usability issue, but I am not sure. I was hoping that if I posted here, we could get some comments from some users.

The scenario is, you have a quiz set to shuffle the order of questions.

On the review page, for different students, each question has a certain number (in the screen-grab below Question 1). However, in the quiz reports, the questions are numbered according to the order on the edit quiz page (in this case Q4).

The difference is most noticable when you click the question score, to open that one question in a pop-up window. There, the link in column Q4 goes to a pop-up saying Quetsion 1. Apparently some teachers find this confusing.

However, that same pop-up can be reached from the quiz review page, by clicking Make comment or override mark.

Screen grabs of the problem

Is this really a problem? if so, what should we do about it?

Mittelwert:  -
Als Antwort auf Tim Hunt

Re: Manual grading pop-up

von Rick Jerz -
Nutzerbild von Particularly helpful Moodlers Nutzerbild von Testers

I had never noticed that!

Interesting problem.

On the quiz report, it seems that it makes sense that a teacher would want to see the overall results for a question.  However, this is somewhat "administrative."

Maybe the quiz report can show results by student, but there should be an administrative "switch" to see it by question.  Hmmm, this could get pretty complex, but maybe worth thinking about.

Mittelwert:  -
Als Antwort auf Tim Hunt

Re: Manual grading pop-up

von Daniel Thies -
Nutzerbild von Core developers Nutzerbild von Plugin developers Nutzerbild von Testers

I have used this and definitely consider the current ordering on the attempts report a feature not a bug. Shuffling implies that each question is independent of the others in the same attempt (order does not matter). If I have essay questions, I would prefer to grade all attempts of a particular question at the same time before going on to a new question. For automatically graded questions it also makes it easy to spot questions which are causing problems for students. If the teacher wants to grade in the order presented, selecting the attempt and using 'override or make comment' works for that.

It might help to have both slot and the shuffle number shown in the popup, but I do not know why. If might also work to allow teachers to toggle the ordering on the report, but I would not do away with the current ordering.



Mittelwert:  -
Als Antwort auf Daniel Thies

Re: Manual grading pop-up

von Tim Hunt -
Nutzerbild von Core developers Nutzerbild von Documentation writers Nutzerbild von Particularly helpful Moodlers Nutzerbild von Peer reviewers Nutzerbild von Plugin developers

I had a thought about this.

Perhaps it would help the teacher to see the original question number (in addition to what the student saw) everywhere. (E.g. on the review page too.)

We could do this by changing the display for any user with mod/quiz:viewreports capability. In the grey 'info' box we would display something like

Question 1

(was 4 before shuffling)
Mittelwert: Useful (2)
Als Antwort auf Tim Hunt

Re: Manual grading pop-up

von Emma Richardson -
Nutzerbild von Documentation writers Nutzerbild von Particularly helpful Moodlers Nutzerbild von Plugin developers

I think that this is the better approach.  The quiz report needs to have all the questions in the same location but I think there is value in knowing what order the student took the questions in as well.  Not sure if you should have it the way you have or reversed - Question 4 (shuffled to position 1) or something similar.

Mittelwert: Useful (1)
Als Antwort auf Tim Hunt

Re: Manual grading pop-up

von Abhilash Mhaisne -

Hi Tim! Thanks for creating the post and images.

I think the order of questions in the report should be the same as original non shuffled order. So when the pop up window appears it should display the question number same as it was before shuffling, not at what number it appeared to a student.

Shuffled questions are for students! I would like to ask everyone here, why display the shuffled number of the question? 

Thanks and Regards,

Abhilash

Mittelwert:  -
Als Antwort auf Abhilash Mhaisne

Re: Manual grading pop-up

von Rick Jerz -
Nutzerbild von Particularly helpful Moodlers Nutzerbild von Testers

Your post made me think about "Why and how do I use this report?"

My most common use is when a student writes me saying "I don't know why I missed questions 4."  I go to this report, see that the student got question 4 wrong, then I click on it to see what the actual question was.

If we show the unshuffled order, how will I know what question 4 was?

So I wonder if the unshuffled order report should be part of statistics reports for the quiz?

Or maybe the "spreadsheet" view should show the unshuffled order, then to see "Question 4" one should click on "Review attempt."  Or maybe there should be one more button (next to, or under "Download") that says "Show unshuffled."

Mittelwert: Useful (1)
Als Antwort auf Rick Jerz

Re: Manual grading pop-up

von Tim Hunt -
Nutzerbild von Core developers Nutzerbild von Documentation writers Nutzerbild von Particularly helpful Moodlers Nutzerbild von Peer reviewers Nutzerbild von Plugin developers

In that scenario, I think that you have to reivew the student's attempt, to see the question numbers as the student saw them. That is one extra click from the quiz reports -> Review page -> Q4. I don't think there is a feasible way to shorten that.

Mittelwert:  -
Als Antwort auf Tim Hunt

Re: Manual grading pop-up

von Rick Jerz -
Nutzerbild von Particularly helpful Moodlers Nutzerbild von Testers

Yep, I can live with this.  So are we approaching agreement that the numbers on this report reflect the "unshuffled" order?  If so, do you think this change will be easy to implement?  Do you want this discussion to go on for a little longer, just to see if anyone finds problems with this new approach?  Does the word "Unshuffled" need to show somewhere on this report?

Mittelwert:  -
Als Antwort auf Rick Jerz

Re: Manual grading pop-up

von Colin Rice -
Hi,

Taking feedback from within my own organisation, it is simply confusing to click on the question in the report and get a different number in the popup box. So I am in the camp that suggests always display unshuffled questions in the report and subsequent popups; shuffled questions are a presentation format for students only, in my honest opinion.


Cheers

Colin

Mittelwert:  -
Als Antwort auf Colin Rice

Re: Manual grading pop-up

von Rick Jerz -
Nutzerbild von Particularly helpful Moodlers Nutzerbild von Testers

Your post makes me wonder if this gets even more confusing when the shuffled, or unshuffled questions are also randomly picked?

Mittelwert:  -
Als Antwort auf Tim Hunt

Re: Manual grading pop-up

von Daniel Thies -
Nutzerbild von Core developers Nutzerbild von Plugin developers Nutzerbild von Testers

Okay in this case you may indeed want to know where question 4 was in the unshuffled order. For example that Q4 for this student is Q10 of the unshuffled questions so you can check on the attempts or statistics report how the rest of the class performed on the same question.

I like the idea of having both visible to the teacher on the popup.

Mittelwert:  -
Als Antwort auf Daniel Thies

Re: Manual grading pop-up

von Tim Hunt -
Nutzerbild von Core developers Nutzerbild von Documentation writers Nutzerbild von Particularly helpful Moodlers Nutzerbild von Peer reviewers Nutzerbild von Plugin developers

My suggestion was to display this to the teacher everywhere the question is displayed. Not to have different displays in different places.

Mittelwert:  -
Als Antwort auf Tim Hunt

Re: Manual grading pop-up

von Rick Jerz -
Nutzerbild von Particularly helpful Moodlers Nutzerbild von Testers

I might be just a little confused and apologize for asking for clarification.

Right now, the question (like #4) is shown on the report and in the "review attempt" of the quiz always as #4, which is #4's "shuffled" position.  You want to make both of these reports use the "unshuffled" position.  Is this what you are saying? One way, or the other way, but not both and not a mixture?

Mittelwert:  -
Als Antwort auf Rick Jerz

Re: Manual grading pop-up

von Daniel Thies -
Nutzerbild von Core developers Nutzerbild von Plugin developers Nutzerbild von Testers

In Tim's example above. Clicking Q1 on the review page and Q4 on the attempts report both link to exactly the same question. If one numbering system is chosen for the display, one of these links will appear to be wrong. One solution is to display both numbers for each question. Another possibility is changing the order of one of the reports so the teacher only sees one set of numbers. However, that will create more difficulty comparing responses from different students or attempts.

Mittelwert:  -
Als Antwort auf Abhilash Mhaisne

Re: Manual grading pop-up

von Abhilash Mhaisne -

Thank you for responding! Reading the discussion here, I can think of  another solution similar to what Tim mentioned :

In the popup window, display the unshuffled question number in front of question number. Add another string below it, say, "Appeared to student on number 4".

So 2 strings (Eg. original question 10, it appeared at no. 4 to a student) :

Question number 10

Appeared to student on number 4

What say?

Also, I have submitted the patch for unshuffled question number to appear on the bug page. But I think I'll wait for a final behaviour which everyone agrees upon.

Mittelwert:  -