Activities: CAPQuiz

mod_capquiz
Maintained by Picture of Sebastian GundersenSebastian Gundersen, Picture of George SchaathunGeorge Schaathun, Picture of André StorhaugAndré Storhaug
CAP is short for Computer Adaptive Practice. In CAPQuiz, the proficiency is measured by a rating. Good answers increase the rating, and bad answers decrease it. To increase the rating, students need to give good answers more of than bad ones over time. Estimating question difficulty is known to be difficult. CAPQuiz automates this process to some extent. The question author must provide an initial estimate, but CAPQuiz improves the estimates based by comparing how the same student answers different questions. Hence the rated question sets will improve over time.
59 sites
41 downloads
16 fans
Current versions available: 1

What is it?

CAP is short for /Computer Adaptive Practice/, a term coined by Klinkenberg, Straatemeier, and van der Maas (2011). Where most LMS quiz systems give the students a fixed sequence of questions regardless of how well the students answer, a CAP system will estimate student ability based on their answers, and try to find questions at the right level of difficulty.

In CAPQuiz, the proficiency is measured by a rating. Good answers increase the rating, and bad answers decrease it. To increase the rating, students need to give good answers more of than bad ones /over time/. We have used CAPQuiz as a mandatory assignment, where the students have to reach a certain rating in order to be allowed to sit the exam.

Estimating question difficulty is known to be difficult. CAPQuiz automates this process to some extent. The question author must provide an initial estimate, but CAPQuiz improves the estimates based by comparing how the same student answers different questions. Hence the rated question sets will improve over time.

Documentation

Documentation is available here, including installation instructions.

History:

The idea of an adaptive learning system at NTNU in Ålesund (then Ålesund University College) was first conceived by Siebe van Albada. His efforts led to a prototype, known as MathGen, written as a standalone server in python.

The first prototype was tested by several lecturers, and was well received by students. There were, however, many problems which we lacked the resources to handle. Most of these problems had already been solved by Moodle and the STACK question type, and it made sense to reimplement the adaptive quiz functionality in Moodle to take advantage of this.

Credits:

Project lead: Hans Georg Schaathun: hasc@ntnu.no

Developers:

Original idea: Siebe Bruno Van Albada siebe.b.v.albada@ntnu.no

The first prototype was funded in part by Norgesuniversitetet.

The development of CAPQuiz has been funded in part by internal grants from Ålesund University College and NTNU Toppundervisning at NTNU - Norwegian University of Science and Technology.

Screenshots

Screenshot #0
Screenshot #1
Screenshot #2
Screenshot #3
Screenshot #4
Screenshot #5
Screenshot #6
Screenshot #7

Contributors

Picture of Sebastian Gundersen
Sebastian Gundersen (Lead maintainer)
Picture of George Schaathun
George Schaathun
Please login to view contributors details and/or to contact them

Comments RSS

Show comments
  • Picture of George Schaathun
    Mon, Mar 25, 2019, 3:56 PM
    Right, I think what you are asking for is for CAPQuiz to issue a grade, which can be recognised as such by other moodle modules/the core.
    It is still beyond my use of Moodle; I use Moodle only for activities, I do not organise taught modules. But obviously you are right. Issuing a grade makes sense, and I have a couple of students who will be working on CAPQuiz and JazzQuiz over the Summer, and this sounds like something we can fix in that timeframe.

    I was just wondering what use case you have in mind. My own use case was to allow the students to continue work on the activity indefinately. There is a deadline for reaching three stars, and I simply record outside moodle who have achieved it soon after the deadline, without disrupting the flow for those heading for five stars.
    How would you organise it (in an ideal world)? Close the activity at the deadline and use the number of stars as a grade? Issue a grade at the deadline and keep the activity open? Or?
    :-- George
  • Picture of Kees Koopman
    Mon, Mar 25, 2019, 11:19 PM

    Hi George,

    Thank you for your response.

    I didn't mean so much that a student gets a grade. That also does not do justice to the intention of CAPQuiz, I think.

    Somewhere in the Moodle database is the student's obtained rating. Joe Smith, for example, has a rating of 1565.

    I have a document, certificate, test, web link or something available for students who have a rating of 1550 and higher. Joe Smith is eligible for this (after all: 1565> 1550).

    In that follow-up document, with Restrict Access there is something like: if the student scores at CAPQuiz> 1550, then he can see the document.

    Writing this I think of a wish.
    At "Activity Completion" >> "Completion tracking" >> "Show activity as complete when conditions are with" >> "Require a rating of ....." (see stash or level up).
    On the dots you can then enter the minimum rating where Moodle automatically ticks the CAPQuiz. This can then be used as input for the Progress Bar.



    You write that you have a deadline for reaching three stars and that you register that outside of Moodle. I like to see the latter automatically and within Moodle smile.
    The Moodle progress bar that I use is a fantastic instrument for that.
    The student may (or must) continue to improve himself; In my opinion, CAPQuiz should simply remain open.

    Regards, Kees Koopman.
  • Picture of Sebastian Gundersen
    Thu, Jun 27, 2019, 9:59 PM
    Hello Kees,

    CAPQuiz now supports the grading system. You can find the documentation here: https://github.com/KQMATH/moodle-mod_capquiz/wiki/Grading
    If you try this feature, feedback is much appreciated.

    Regards, Sebastian Gundersen.
  • Picture of George Schaathun
    Mon, Jul 1, 2019, 7:01 PM
    Thank you, Sebastian, well done.
    I would be grateful, Kees, if you would tell me if the current solution meets your requirement. What we do is to export the number of stars to the gradebook, until the deadline.
    Since I personally only manage single activities, and not complete courses, I do not really know what's needed.
    :-- George
  • Picture of George Schaathun
    Thu, Oct 24, 2019, 8:43 PM
    FYI. We have detected a bug in the update of question ratings. We believe that we have found the fix, and as soon as we have double-checked it, we will make a new release. Anyone using the current release should set the question k-factor to zero. The result of the bug is that questions are pushed towards extreme ratings, so almost all questions have either very high or very low ratings. If the k-factor is zero, ratings do not change.
  • Picture of George Schaathun
    Thu, Oct 24, 2019, 8:44 PM
    BTW student ratings are not affected.
  • Picture of George Schaathun
    Mon, Oct 28, 2019, 4:10 PM
    This problem is resolved in version 0.4.1. Please upgrade if you want dynamic problem ratings.
  • Picture of Andy Gokce
    Sat, Nov 23, 2019, 9:33 AM
    Hi,
    The screenshots show that the teacher needs to select the number of questions in the quiz (10 in the example). But you say in the description that students had to reach a certain rating in order to be allowed to sit the exam (how you used it). So, how does that work? Can the students take the same capquiz multiple times?

    Or, Is there a way to not limit the number of questions in a quiz and require them continue solving problems until they reach a required rating?

    Thanks,
  • Picture of George Schaathun
    Sat, Nov 23, 2019, 8:02 PM
    Right, I think there are two separate issues here. One is the design of the CAPQuiz activity, and the other is the role of the CAPQuiz activity in the module design. CAPQuiz is not designed as an assessment exercise, merely as a learning exercise.

    1. In the module design, it is quite common in Norwegian HE to have compulsory assignments, which do not count towards the exam, but which are prerequisites to sit the exam. This is how my colleagues and I intended CAPQuiz to be used. Through the CAPQuiz activity the student earns a number of stars. We can require that the student achieve a certain number of stars as their compulsory assignment, which has to be achieved before being allowed to sit the exam. We have not considered if the number of stars could be factored into the final grade in a sensible way, and the plugin should mature beyond the current experimental stage before such use is considered in practice.

    2. In the activity design, there is no limit on the number of questions. Obviously only a finite number of questions is added to the activity at any time. Because each question has to be assigned a rating, CAPQuiz will not automatically draw random questions from the question bank. It is possible to bulk add questions with the same starting rating, but I cannot remember how easy this is in the UI. There is no limit to the number of questions the students get, they will keep getting questions as long as they want to, usually repeating questions if they do not rapidly progress.

    Does this answer you question?

    Repeating questions is not a concern with parameterised questions in random variants as supported by STACK and Calculated. Other static question types will of course require an enormous number of questions to make repetitions few and far between. This will work out differently in different subjects.

    Now, it should be noted that there is a performance limit in the moodle question bank. This is no problem for small quizzes or quizzes using light-weight question types, but with CAPQuiz we have been expecting quizzes with a large number of STACK questions in many deployed variants, and the combination of a computationally costly question type (STACK) and the large number of questions (40+ in 100+ variants), this broke down. We had one very successful student assignment without STACK questions, but I had to cancel the second one where STACK was used because of performance issues. Hopefully we can resolve this issue over the next six months.
  • Picture of Andy Gokce
    Sun, Nov 24, 2019, 10:40 AM
    Thanks so much for the detailed answer. This is an awesome plugin. I guess I should install and test it but I am a little confused. I guess I need to be more specific:

    Let’s say a students takes a capquiz activity which has 10 problems to solve and finishes the test but doesn’t reach the required level (the rating or the stars) to pass (like the rating you require them to be able sit an exam). What happens? Do they take it again until they pass?

    And if there are a large number of questions in a capquiz activity (say 100), can the student leave the test when they are tired and continue where they left off the next day?

    Thanks a lot
  • Picture of George Schaathun
    Sun, Nov 24, 2019, 2:48 PM
    A CAPQuiz activity never ends, and the student can never start over from scratch (unless the teacher manually resets their rating). Instead the student can always get another question and lose or gain rating points. And yes, if the students leave, they would return where they left off.

    When I use it as a compulsory assignment, the requirement is that the student achieve three stars at any point before the deadline. While skilled students can do this in less than an hour, the assumption is that most students will have to try, fail, return to the book, ask a friend, and get back to CAPQuiz to continue another evening. But this obviously depends a lot on the topic and the question design.
  • Picture of George Schaathun
    Sun, Nov 24, 2019, 2:49 PM
    Thanks for the encouraging words too.
  • Picture of Andy Gokce
    Mon, Nov 25, 2019, 1:22 AM
    Perfect! Thanks so much!
  • Picture of Andy Gokce
    Fri, Dec 13, 2019, 7:17 PM
    Hi,
    We installed the plugin, created our first capquiz but for some reason, there is no way to add questions. We have a lot of questions in our question bank but there is no option to add any questions in the settings or anywhere else. Also, I got an error saying that it cannot write on database when I tried to update the settings.
    Thanks,
  • Picture of George Schaathun
    Sat, Dec 14, 2019, 4:53 PM
    Thanks for your input.
    That error message does appear for us too even when everything succeeds. We are working on this, but I think it is unrelated to your real problem. Either way, I need to know more about what you do and what happens.

    When you create the CAPQuiz (after the standard config screen), you should get a dialog with a button to «Create Question List» (if you have previously created templates, you can choose from them as well, but the create button is always there). Is this what you see? What happens when you hit the create button?

    When a question list has been created or chosen, you get a multi-tab config interface. To add questions, choose the «Questions» tab (third one from the left I think). Here you see the chosen questions on the left and a question bank interface on the right, incl. a new question button, and a drop-down to choose a category in the bank. Do you see this?

    When you get the DB write error, which settings screen are you using? Via the gear menu? One of the tabs on the main screen?

    BTW. If you post bugs and issues on github (https://github.com/KQMATH/moodle-mod_capquiz) instead, all the developers will see it too. I try to pay attention to this forum as well, but I need to move the discussion to github when I find that it requires action on our part. If you don't want to be on github, that's fine, but it gives you the opportunity to follow the issue all the way through.

1 2
Please login to post comments