CAPQuiz
Sebastian Gundersen, 
George Schaathun, What is it?
CAP is short for /Computer Adaptive Practice/, a term coined by Klinkenberg, Straatemeier, and van der Maas (2011). Where most LMS quiz systems give the students a fixed sequence of questions regardless of how well the students answer, a CAP system will estimate student ability based on their answers, and try to find questions at the right level of difficulty.
In CAPQuiz, the proficiency is measured by a rating. Good answers increase the rating, and bad answers decrease it. To increase the rating, students need to give good answers more of than bad ones /over time/. We have used CAPQuiz as a mandatory assignment, where the students have to reach a certain rating in order to be allowed to sit the exam.
Estimating question difficulty is known to be difficult. CAPQuiz automates this process to some extent. The question author must provide an initial estimate, but CAPQuiz improves the estimates based by comparing how the same student answers different questions. Hence the rated question sets will improve over time.
Documentation
Documentation is available here, including installation instructions.
See the migration guide for details on how to update to a newer version of CAPQuiz.
History:
The idea of an adaptive learning system at NTNU in Ålesund (then Ålesund University College) was first conceived by Siebe van Albada. His efforts led to a prototype, known as MathGen, written as a standalone server in python.
The first prototype was tested by several lecturers, and was well received by students. There were, however, many problems which we lacked the resources to handle. Most of these problems had already been solved by Moodle and the STACK question type, and it made sense to reimplement the adaptive quiz functionality in Moodle to take advantage of this.
Credits:
Project lead: Hans Georg Schaathun: hasc@ntnu.no
Developers:
- Aleksander Skrede aleksander.l.skrede@ntnu.no
- Sebastian S. Gundersen sebastian@sgundersen.com
- André Storhaug andr3.storhaug@gmail.com
Original idea: Siebe Bruno Van Albada siebe.b.v.albada@ntnu.no
The first prototype was funded in part by Norgesuniversitetet.
The development of CAPQuiz has been funded in part by internal grants from Ålesund University College and NTNU Toppundervisning at NTNU - Norwegian University of Science and Technology.
Screenshots
Contributors
CAPQuiz now supports the grading system. You can find the documentation here: https://github.com/KQMATH/moodle-mod_capquiz/wiki/Grading
If you try this feature, feedback is much appreciated.
Regards, Sebastian Gundersen.
I would be grateful, Kees, if you would tell me if the current solution meets your requirement. What we do is to export the number of stars to the gradebook, until the deadline.
Since I personally only manage single activities, and not complete courses, I do not really know what's needed.
:-- George
The screenshots show that the teacher needs to select the number of questions in the quiz (10 in the example). But you say in the description that students had to reach a certain rating in order to be allowed to sit the exam (how you used it). So, how does that work? Can the students take the same capquiz multiple times?
Or, Is there a way to not limit the number of questions in a quiz and require them continue solving problems until they reach a required rating?
Thanks,
1. In the module design, it is quite common in Norwegian HE to have compulsory assignments, which do not count towards the exam, but which are prerequisites to sit the exam. This is how my colleagues and I intended CAPQuiz to be used. Through the CAPQuiz activity the student earns a number of stars. We can require that the student achieve a certain number of stars as their compulsory assignment, which has to be achieved before being allowed to sit the exam. We have not considered if the number of stars could be factored into the final grade in a sensible way, and the plugin should mature beyond the current experimental stage before such use is considered in practice.
2. In the activity design, there is no limit on the number of questions. Obviously only a finite number of questions is added to the activity at any time. Because each question has to be assigned a rating, CAPQuiz will not automatically draw random questions from the question bank. It is possible to bulk add questions with the same starting rating, but I cannot remember how easy this is in the UI. There is no limit to the number of questions the students get, they will keep getting questions as long as they want to, usually repeating questions if they do not rapidly progress.
Does this answer you question?
Repeating questions is not a concern with parameterised questions in random variants as supported by STACK and Calculated. Other static question types will of course require an enormous number of questions to make repetitions few and far between. This will work out differently in different subjects.
Now, it should be noted that there is a performance limit in the moodle question bank. This is no problem for small quizzes or quizzes using light-weight question types, but with CAPQuiz we have been expecting quizzes with a large number of STACK questions in many deployed variants, and the combination of a computationally costly question type (STACK) and the large number of questions (40+ in 100+ variants), this broke down. We had one very successful student assignment without STACK questions, but I had to cancel the second one where STACK was used because of performance issues. Hopefully we can resolve this issue over the next six months.
Let’s say a students takes a capquiz activity which has 10 problems to solve and finishes the test but doesn’t reach the required level (the rating or the stars) to pass (like the rating you require them to be able sit an exam). What happens? Do they take it again until they pass?
And if there are a large number of questions in a capquiz activity (say 100), can the student leave the test when they are tired and continue where they left off the next day?
Thanks a lot
When I use it as a compulsory assignment, the requirement is that the student achieve three stars at any point before the deadline. While skilled students can do this in less than an hour, the assumption is that most students will have to try, fail, return to the book, ask a friend, and get back to CAPQuiz to continue another evening. But this obviously depends a lot on the topic and the question design.
We installed the plugin, created our first capquiz but for some reason, there is no way to add questions. We have a lot of questions in our question bank but there is no option to add any questions in the settings or anywhere else. Also, I got an error saying that it cannot write on database when I tried to update the settings.
Thanks,
That error message does appear for us too even when everything succeeds. We are working on this, but I think it is unrelated to your real problem. Either way, I need to know more about what you do and what happens.
When you create the CAPQuiz (after the standard config screen), you should get a dialog with a button to «Create Question List» (if you have previously created templates, you can choose from them as well, but the create button is always there). Is this what you see? What happens when you hit the create button?
When a question list has been created or chosen, you get a multi-tab config interface. To add questions, choose the «Questions» tab (third one from the left I think). Here you see the chosen questions on the left and a question bank interface on the right, incl. a new question button, and a drop-down to choose a category in the bank. Do you see this?
When you get the DB write error, which settings screen are you using? Via the gear menu? One of the tabs on the main screen?
BTW. If you post bugs and issues on github (https://github.com/KQMATH/moodle-mod_capquiz) instead, all the developers will see it too. I try to pay attention to this forum as well, but I need to move the discussion to github when I find that it requires action on our part. If you don't want to be on github, that's fine, but it gives you the opportunity to follow the issue all the way through.
I have tried to type up documentation for this here: https://github.com/KQMATH/moodle-mod_capquiz/wiki/Grading
Please let me know if it answers your questions.