Hi all,
I've recently received a question from our faculty if we would be able to provide the same analysis tools for quiz attempts and questions as they have in dedicated platforms for eAssessment. So far, I've covered most of their questions, but there are a few things that remain unclear or impossible:
p-value
This is supposed to represent the degree of difficulty of a question, and is calculated as the overall average score for any given question. Is that the same as the Facility index calculated in Moodle?
Rit and Rir
These are Pearson product moment correlation coefficient values, but somewhat different from the way Moodle uses that. Rit represents the result for a specific question in relation to the quiz result as a whole (including the specific question), Rir represents the result for a specific question in relation to the rest of the questions in the same quiz.
This kind of sounds like the Discrimination Index calculation, but with a twist.
a-value
This represents the quality of each distractor expressed as a value between 0 and 1 (or a percentage) for how many of the participants selected that distractor in their quiz/question attempt. E.g. if a question has option B set as the only correct distractor of 4 options, and 38% of the participants selected it, the a-value for this distractor would be 0.38. The sum of the a-values for the other 3 distractors would be 1 - 0.38 = 0.62.
Rat and Rar
Similar to Rit and Rir, but used to analyse distractors in closed questions. These would return values for each distractor. This is an option I haven't yet found in Moodle.
Most of this information can somehow be collected from the Moodle database, but I haven't found a tool to aggregate all these into a nice and user friendly report.