Discrimination index under Moodle quiz

Re: Discrimination index under Moodle quiz

by Tim Hunt -
Number of replies: 2
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
The formulas that Moodle uses are on https://docs.moodle.org/dev/Quiz_statistics_calculations and https://docs.moodle.org/dev/Quiz_statistics_calculations_in_practice.

When this was first implemented, my colleague Phil Butcher spent a lot of time comparing the numbers Moodle calculated with independently calculated correct values.

And, there is an automated test of these calculations, so we would konw if something unexpectedly changed and broke the calculations. The test data for that is here https://github.com/moodle/moodle/tree/master/mod/quiz/report/statistics/tests/fixtures (in a rather cryptic format).

If you are sure that the calculations are wrong, please try to make the simplest possible example that shows the bug (e.g. can you make it happen on a quiz with 2-3 questions and 3-5 attempts?) If so, please give us specific details. Thanks.
Average of ratings: Useful (2)
In reply to Tim Hunt

Ynt: Re: Discrimination index under Moodle quiz

by Selcen Yüksel -

Hi,

As Ada states, there seems a problem about the calculation of discrimination index is given at the link https://docs.moodle.org/dev/Quiz_statistics_calculations#Discrimination_index . When I manually calculate the discrimination index, I did not find the same result as I got from in Moodle quiz statistics. I read most of the topics related with discrimination index, but couldn't find a clear answer. I add the xls file that includes my calculations. Is there anyone to explain it clearly?

I have another question: I cannot see the item difficulty and discrimination index values near the questions at item bank. Can I provide this?

Thank you,

Kind regards

Selcen


Average of ratings: -
In reply to Selcen Yüksel

Re: Ynt: Re: Discrimination index under Moodle quiz

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

When the statistics calculations were first implemented a long time ago, one of my colleagues tested them very carefully.

Also, Moodle is open source. If you really want to know what is going on, you can read the code and check it. It's not the easiest code to follow (because it was written with an eye to efficiency of the calculation as well as clarity) but this seems to be where it computes the discrimination index: https://github.com/moodle/moodle/blob/87afa4d7595eeafdf9e3f83d53272ee61fb66780/question/classes/statistics/questions/calculator.php#L415

The key difference you can get with calculations of discrimination index is whether you are computing the covariance between

  • the mark for this question and the total for all questions, or
  • the mark for this question and the total for the other questions that are not this one.

Moodle uses the second options (if my memory is right). That bit of code seems to be here: https://github.com/moodle/moodle/blob/87afa4d7595eeafdf9e3f83d53272ee61fb66780/question/classes/statistics/questions/calculator.php#L387

Average of ratings: -