I have been scouring Moodle documentation and forums and can't find any reference to form equating.
I want to construct high-stakes certification exams on Moodle, or at least I want to see how far I can advance this before I have to turn to a third-party exam delivery service. If there is a suitable plug-in for certification exams that would be interesting.
I already know Moodle 4 can construct quizzes using item banks. I need to be able to equate each unique quiz form against set parameters for item difficulty level, either just before it is presented to the student (and then reshuffled until it fits the parameters), or alternatively, I need Moodle to deliver the exam and then perform the equating on the student's raw score.
I have never seen what you describe as part of Moodle quizzing, but I am not certain I understand what you are describing. I just searched and found this
https://en.wikipedia.org/wiki/Equating
https://en.wikipedia.org/wiki/Equating
Marcus,
Thank you for responding. Yes, this is what I mean by equating. I am guessing from the sound of crickets chriping
that Moodle's off-the-shelf product doesn't have this feature. There are other specialty item banking and construction companies out there (I have reviewed three) but these require us to let their software assemble and deliver the exam on-the-fly. Otherwise I have to create static "fixed" forms (i.e. multiple versions to achieve equating.
Callum
Thank you for responding. Yes, this is what I mean by equating. I am guessing from the sound of crickets chriping
Callum
I think you interpretation of the auditory feedback is correct. In my view the Moodle quiz engine is the most powerful available anywhere in terms of question types and the ways that the questions are presented, but the emphasis is more on learning that complexity of grading. It might be possible to create/develop something like what you are describing but that would take time and money. A standard Moodle quiz is a static thing, i.e. you assemble the questions you want to ask, including some that may be drawn at random and the students take the quiz.
I am hearing the term equating for the first time. Does it mean that the quiz marking scheme should recognize \( x^2 - 1 \) and \( (x-1)*(x+1) \) and \( -1 + x^2 \) and \( (x+1)*(x-1) \) and many more as equal and mark as correct?
The the additional question type STACK can do that. The back end of STACK is the popular open source CAS Maxima. The front end is MathJax. As you see, it has so many parts, will look daunting at the beginning. If you are fit in basic mathematical thinking, suddenly it all makes sense.
I'm surprised that you didn't find it in here at moodle.org. Just check the first five discussions in the Mathematical tools forum.
I don't claim to know more than the abc. Had the great opportunity of meeting a full delegation of STACK cracks including its inventor two weeks ago at the MoodleMoot DACH 2023. I think the presence of the Guru makes a difference.
Note to the moderator: Take this discussion to the Mathamatics tools forum, where all the math cracks are gathered.
Hi Visvanath,
Thank you for your detailed response. I will take a look around.
I imagined that this would be so fundamental to building fair exams that someone would have already developed a plug in for Moodle. I am concerned that if I am struggling to find someone familiar this this issue that, as powerful as Moodle is, it doesn't have this functionality.
Callum
Thank you for your detailed response. I will take a look around.
I imagined that this would be so fundamental to building fair exams that someone would have already developed a plug in for Moodle. I am concerned that if I am struggling to find someone familiar this this issue that, as powerful as Moodle is, it doesn't have this functionality.
Callum
Hi
You're welcome!
I don't know how fundamental this feature is. What are the other comparable MC systems, which has something similar to STACK?
You're welcome!
I don't know how fundamental this feature is. What are the other comparable MC systems, which has something similar to STACK?
If my interpretation of what you are asking for is correct neither Moodle or any Moodle plugin has this feature. I have been reading posts on these forums most days for 20 years and have not seen a request for this (Visvanath has been around here for a similar time) . But I am not clear I have understood your requirement.
Hi Callum
To what Marcus says, a couple of sample questions and answers are needed from you to be clear what we are talking about.
To what Marcus says, a couple of sample questions and answers are needed from you to be clear what we are talking about.
Marcus,
Thank you. This means I have to
a) rely on a 3rd-party software provider to use one of their proprietary programs, or
b) hire someone to develop a plug-in, or
c) abandon the use of question banks in favor of using "fixed forms" (e.g. version 1, 2, 3, etc. of an exam)
If I have to pursue c), do you know if there is a such a function within Moodle that will ensure that students are presented a different exam version in subsequent attempts? We're trying to avoid them learning the exam questions through repeating the exam. If yes, is there any kind of control over what exam gets presented to the student?
Thanks,
Callum
Thank you. This means I have to
a) rely on a 3rd-party software provider to use one of their proprietary programs, or
b) hire someone to develop a plug-in, or
c) abandon the use of question banks in favor of using "fixed forms" (e.g. version 1, 2, 3, etc. of an exam)
If I have to pursue c), do you know if there is a such a function within Moodle that will ensure that students are presented a different exam version in subsequent attempts? We're trying to avoid them learning the exam questions through repeating the exam. If yes, is there any kind of control over what exam gets presented to the student?
Thanks,
Callum
As others have said, Moodle does not have built in the feature to give students different variants of a test, and then re-calibrate the results from each variant to make the marks comparable (at least according to some statistical theory).
However, Moodle does offer a lot of the parts:
1. You can create different variants of the test for different students. Just create the differnt quizzes, then use conditional availability (e.g. based on group) to control who sees each one.
1b. However, what most people do is to create one test, where each question has a number of variants. That way, there is even more variability between different student's quizzes, making cheating harder, if that is your goal. E.g. if you have 10 questions, and 3 variants of each question, that is 3^10 ~= 59049 possible quizzes - but it makes the kind of statistics you want to do harder.
2. After a quiz with randomisation has been run, the report at Quiz -> Results -> Statistics provdies quite a lot of high quality information about what happened, based on 'Classical test theory'. Probably enough to make it a simple task to compute what score adjustmenents you want to make to bring the scores from each quiz variant into line.
2b. Alternatively, if you are using variants of each question, then by comparing the computed stats, especially 'Facility index' for each variant of the question, you can see if all the different variants turned out to be about equally difficult.
3. In the Moodle gradebook, you can control how the different quiz scores are adjusted before being added to determine the course total. So, if you manually compute the adjustments required in step 2, then you can get Moodle to apply them.
However, this is more work that a system that specifically supports this.
Also, my feeling is, that to acutally do this in a statistically meaningful way, you are going to need a very large number of students (my guess would be at least 10s of thousands).
However, Moodle does offer a lot of the parts:
1. You can create different variants of the test for different students. Just create the differnt quizzes, then use conditional availability (e.g. based on group) to control who sees each one.
1b. However, what most people do is to create one test, where each question has a number of variants. That way, there is even more variability between different student's quizzes, making cheating harder, if that is your goal. E.g. if you have 10 questions, and 3 variants of each question, that is 3^10 ~= 59049 possible quizzes - but it makes the kind of statistics you want to do harder.
2. After a quiz with randomisation has been run, the report at Quiz -> Results -> Statistics provdies quite a lot of high quality information about what happened, based on 'Classical test theory'. Probably enough to make it a simple task to compute what score adjustmenents you want to make to bring the scores from each quiz variant into line.
2b. Alternatively, if you are using variants of each question, then by comparing the computed stats, especially 'Facility index' for each variant of the question, you can see if all the different variants turned out to be about equally difficult.
3. In the Moodle gradebook, you can control how the different quiz scores are adjusted before being added to determine the course total. So, if you manually compute the adjustments required in step 2, then you can get Moodle to apply them.
However, this is more work that a system that specifically supports this.
Also, my feeling is, that to acutally do this in a statistically meaningful way, you are going to need a very large number of students (my guess would be at least 10s of thousands).
Tim,
Thank you for your detailed response. I apologize for not responding sooner. You have thought this through carefully and I am still learning about Moodle. I suspect that I am going to need to hire someone to create a custom algorithm to accomplish what we want. I know there are item banking and test assembly companies out there that offer solutions. While I don't mind paying for code development, I'm not inclined to pay a third party for every single exam I deliver in perpetuity. I will investigate further and who knows, maybe we will be the ones to bring this feature to Moodle! We've done other interesting work like this before.
Callum
Thank you for your detailed response. I apologize for not responding sooner. You have thought this through carefully and I am still learning about Moodle. I suspect that I am going to need to hire someone to create a custom algorithm to accomplish what we want. I know there are item banking and test assembly companies out there that offer solutions. While I don't mind paying for code development, I'm not inclined to pay a third party for every single exam I deliver in perpetuity. I will investigate further and who knows, maybe we will be the ones to bring this feature to Moodle! We've done other interesting work like this before.
Callum