Quiz

Where to see the individual question statistics for randomized questions?

 
Picture of Samuli Karevaara
Where to see the individual question statistics for randomized questions?
Core developersTranslators

Apologies if this is being asked frequently, I tried searching the forums, the tracker and the wiki, but the workaround mentioned here https://moodle.org/mod/forum/discuss.php?d=316792 and here https://tracker.moodle.org/browse/MDL-48092 doesn't seem to work anymore in 3.4.

I have a Quiz with just two randomized questions from the question bank, but the stats at /mod/quiz/report.php?id=ID&mode=statistics only gives stats for "question in pos #1" and "question in pos #2". I don't know which question is which as these are randomized. Previously (mentioned in the links above) the link to the latter question details showed a table with all the question bank questions that the quiz is utilizing, but it doesn't seem to be so anymore in 3.4.

Where can I see the discrimination and other similar statistics for the questions in the questionbank when they are randomly shown on the quiz?

 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: Where to see the individual question statistics for randomized questions?
Core developersDocumentation writersParticularly helpful MoodlersPlugin developers

This has not changed between Moodle 3.3 and 3.4.

What does make a difference is whether your category of questions contains 10 or 11 questions. Once there are more than 10 different questions to display related to position 1, we don't try to show them all on the front page of the report. We just show a summary, and then you have to click a link ot drill down into the details (the report MDL-48092 is about the fact that working out which link to click is too difficult).

 
Average of ratings: Useful (1)
Picture of Samuli Karevaara
Re: Where to see the individual question statistics for randomized questions?
Core developersTranslators

Ok, in the tracker issue versions 2.7 and 2.8 are mentioned, that's why I thought this has changed or been improved as far as the UI goes.

I have a test category of three questions. I'm showing two of them in a Quiz at random. Now when I go to the Quiz statistics page, I'm shown discrimination index et cetera for "question #1" and for "question #2". Both links go to a similar drill down for each question. But neither of them has any indication as to what question out of the three in the category it is. How is the discrimination calculated, as it needs to be question specific, not "slot specific"?

 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: Where to see the individual question statistics for randomized questions?
Core developersDocumentation writersParticularly helpful MoodlersPlugin developers

The key bit in the tracker is where it says 'Open' and 'Unresolved'. Those versions mostly give you a clue of when this was reported.

Can you show us a screen-grab of the report you are seeing? It certainly should contain the stats for each of the three questions.

 
Average of ratings: -
Picture of Samuli Karevaara
Re: Where to see the individual question statistics for randomized questions?
Core developersTranslators

Thanks for your answers. I'll add screenshots here. Going to Quiz settings > Results > Statistics gives this:

Quiz statistics screenshot - 1


Then, clicking on either of the links gives a screen similar to this:

Quiz statistics screenshot - 2


I don't get a different page by clicking the last question, they are both showing stats for the "slot #".

 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: Where to see the individual question statistics for randomized questions?
Core developersDocumentation writersParticularly helpful MoodlersPlugin developers

It is saying that only two attempts have been made at the quiz. Therefore, for most questions, there is nothing to show, and so the report does not show anything.

(That is not just a facetious remark. It is also an accurate description of how the report works.)

 
Average of ratings: -
Picture of Devon Ritter
Re: Where to see the individual question statistics for randomized questions?
 

We recently updated to 3.4, and something does appear to have broken for our quiz statistics as a result.

Specifically, most of our exams are designed to pull random questions from categories. In both the online statistics report view, and the downloaded files however, statistics for the first category of random questions no longer get displayed, but instead, just the fact that random questions are pulled from that category X # of times. This is the case whether 1 question is to be randomly pulled from a category, or 10+.

Screenshots as examples:

In the below, the quiz is set to pull 1 question from a bank of 4 for Q1, and 1 question from a bank of 4 for Q2. For Q2, results display as expected, but for Q1, no breakdown of results is accessible.



In the below, the quiz is set to pull 13 questions from a bank of 25 for Unit 1, and 7 questions from a bank of 15 for Unit 2. Based on previous report versions, I would have expected to see the beginning of the actual statistics for Unit 1 right after the Q13 line. The report now skips over any/all data for the first category that gets questions randomly assigned, and then immediately provides data for subsequent categories after every question slot. 


Question slots 14 and 15 both pull from the same category (Unit 2). Individual item response data is the same for all 15 questions, and that data gets repeated 7 times in the downloaded report (once for each number of questions randomly pulled from that category). The only unique characteristics for the 2nd, 3rd... categories is that it calculates the statistics for each question slot individually (for instance, students answered Q14 correctly 60% of the time, and Q15 65% of the time).



The end result is that we now both lack any real useful data on whatever questions happen to be in the first randomly assigned category (the 4 questions from example 1 above, and the 25 questions from example 2), and in downloaded reports, have useful, but quite a bit of redundant data for all remaining randomly pulled from categories.

On quizzes where Q1 is not randomly assigned, the statistics report does pull data for that question accurately.

 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: Where to see the individual question statistics for randomized questions?
Core developersDocumentation writersParticularly helpful MoodlersPlugin developers

Are Q1 and Q2 pulling two questions from the same bank of 4 questions? That is what it looks like from the screen grab.

In that case, the statistics are the same whichever position the question appeared in, so the break-down is only shown once.

Therefore, what you see is expected, and has not changed in any recent Moodle releases.

 
Average of ratings: -
Picture of Devon Ritter
Re: Where to see the individual question statistics for randomized questions?
 

No, Q1 refers to one bank of 4 questions; Q2 refers to a seperate bank of entirely different questions, so we are not getting any data for the 4 questions in the Q1 bank (or in the case of my second quiz use case example, the 25 questions in it's Unit 1 category).

 
Average of ratings: -