Facility index over 100%?

Facility index over 100%?

由Q Peiffer發表於
Number of replies: 7

Hi folks, hoping to find some transparency in how the Facility index field gets calculated!  Currently on Moodle 4.3.5+ (Build: 20240719) at NC State.

In some courses using Quizzes with Random slots, multiple choice questions get a facility index over 100%. This can vary between 102-117% as a reported facility index for a certain question.

Anyone know how this gets calculated in more detail, why facility index might be above 100%, or if there's any documentation I can look at?  MoodleDocs on Quiz results and the icma statsitics doc don't point at any examples where facility index can be above 100.  is there a bug?

Thanks kindly

評比平均分數： -
In reply to Q Peiffer

Re: Facility index over 100%?

由Tim Hunt發表於
Core developers的相片 Documentation writers的相片 Particularly helpful Moodlers的相片 Peer reviewers的相片 Plugin developers的相片
I'm pretty sure that is a bug.

Would be really interesting to work out why and how that is happening.
評比平均分數：Useful (1)
In reply to Tim Hunt

Re: Facility index over 100%?

由Q Peiffer發表於
Let me know if I can provide any info on that. I'm restricted by FERPA, but I can escalate to our in-house Moodle devs and see what they have to say about it!
評比平均分數： -
In reply to Tim Hunt

Re: Facility index over 100%?

由Q Peiffer發表於
Whoever developed this Quiz managed to add more Random question slots to the Quiz than there are questions in the question bank category at this time.  It looks like the questions may have been located in the category that the Random slots pointed at, but were moved out of the category and into a different category later.  Given this, do you think that the Facility index score could have been recalculated with a different weight for the actual points of the question/maximum points a student could have earned on that question across all attempts?  Is it related to the category?
評比平均分數： -
In reply to Q Peiffer

Re: Facility index over 100%?

由Tim Hunt發表於
Core developers的相片 Documentation writers的相片 Particularly helpful Moodlers的相片 Peer reviewers的相片 Plugin developers的相片
I don't think moving questions in the question bank, or editing the default mark of teh questoin in the question bank, would affect existing quiz attempts. So, I don't think this explains it.
評比平均分數：Useful (1)
In reply to Tim Hunt

Re: Facility index over 100%?

由Brett Dalton發表於
Moodle HQ的相片 Particularly helpful Moodlers的相片
My suspsion is it could be random questions causing problems with the way things are calculated. But thats based more on know some of the other complexity around random questions rather than anything else.
評比平均分數：Useful (1)
In reply to Tim Hunt

Re: Facility index over 100%?

由Q Peiffer發表於

I gotta find the math where this facility index score is calculated and identify what it actually averages over.

評比平均分數： -
In reply to Q Peiffer

Re: Facility index over 100%?

由Tim Hunt發表於
Core developers的相片 Documentation writers的相片 Particularly helpful Moodlers的相片 Peer reviewers的相片 Plugin developers的相片
There should be an explanation at https://docs.moodle.org/dev/Quiz_statistics_calculations, but currently equation rendering is broken there, which makes the page not very helpful. (MDLSITE-8046).

The code that computes the statistics is spread over various classes, but if you want to look, https://github.com/moodle/moodle/blob/3e73c33f9920fbb066f2c554fb998af46d4706d8/question/classes/statistics/questions/calculator.php#L37 is probably a reasonable place to start. (And, how that is called from mod/quiz/report/statistics code.)
評比平均分數： -