Question types ::: qtype_coderunner
Maintained by Richard Lobb, TimTim Hunt
A question type that allows question authors to set programming questions in which the student answer is code in some programming language, which is graded by running it. More generally it can handle any question to which the answer is text that can be graded by a computer program written by the question author.
Latest release:
3051 sites
123 fans
Current versions available: 4

CodeRunner is a Moodle question type that allows teachers to run a program in order to grade a student's answer. By far the most common use of CodeRunner is in programming courses where students are asked to write program code to some specification and that code is then graded by running it in a series of tests. CodeRunner questions have also been used in other areas of computer science and engineering to grade questions in which many different correct answers are possible and a program must be used to assess correctness. 

Regardless of the behaviour chosen for a quiz, CodeRunner questions always run in an adaptive mode, in which students can click a Check button to see if their code passes the tests defined in the question. If not, students can  resubmit, typically for a small penalty. In the typical 'all-or-nothing' mode, all test cases must pass if the submission is to be awarded any marks. The mark for a set of questions in a quiz is then determined primarily by which questions the student is able to solve successfully and then secondarily by how many submissions the student makes on each question. However, it is also possible to configure CodeRunner questions so that the mark is determined by how many of the tests the code successfully passes.

CodeRunner has been in use at the University of Canterbury for over seven years, running millions of student quiz question submissions in Python, C , JavaScript, PHP, Octave and Matlab. Laboratory work, assignment work and mid-semester tests in the introductory first year Python programming course (COSC121), which has around 650 students in the first semester and 350 in the second, are all assessed using CodeRunner questions. The final exams for COSC121 have also been run using Moodle/CodeRunner since November 2014. Other courses at the University of Canterbury using CodeRunner include:

  1. ENCE260 Computer Systems
  2. ENCN305 Programming, Statistics and Optimisation
  3. EMTH171 Mathematical Modelling and Computation
  4. SENG02 Software Engineering I
  5. COSC261 Formal Languages and Compilers
  6. COSC 262 Algorithms
  7. COSC367 Computational Intelligence
  8. ENCE360 Operating Systems
  9. SENG365 Web Computing Architectures

CodeRunner is also being used at over 600 other sites worldwide.

CodeRunner currently supports Python2 (considered obsolescent), Python3, C, C++, Java, PHP, JavaScript (NodeJS), Octave and Matlab. The architecture allows easy extension to other languages.

CodeRunner can safely be used on an institutional Moodle server, provided that the sandbox software in which code is run ("Jobe") is installed on a separate machine with adequate security and firewalling. However, if CodeRunner-based quizzes are to be used for tests and final exams, a separate Moodle server is recommended, both for load reasons and so that various Moodle communication facilities, like chat and messaging, can be turned off without impacting other classes.

The CodeRunner question type can be installed on any modern Moodle system (version 3.0 or later), on Linux, Windows and Mac. For security reasons submitted jobs are run on a separate machine called the "Jobe server" or "Jobe sandbox machine". CodeRunner is intitially configured to use a small outward-facing Jobe server at the University of Canterbury, and this can be used for initial testing.  However, this is not suitable for production use, for which institutions will need to install their own Jobe server. Instructions for installing a Jobe server are given in the Jobe documentation. Once Jobe is installed, use the Moodle administrator interface for the CodeRunner plug-in to specify the Jobe host name and perhaps port number.

A single 4-core Moodle server can handle an average quiz question submission rate of about 60 quiz questions per minute while maintaining a response time of less than about 3 - 4 seconds, assuming the student code itself runs in a fraction of a second. We have run CodeRunner-based exams with nearly 300 students and experienced only light to moderate load factors on an 8-core Moodle server. The Jobe server, which runs student submissions (see below), is even more lightly loaded during such an exam.

The full documentation for CodeRunner, together with forums, questions banks and other resources, is at

Potential privacy issues

None that I'm aware of.


Screenshot #0
Screenshot #1
Screenshot #2
Screenshot #3
Screenshot #4
Screenshot #5
Screenshot #6
Screenshot #7
Screenshot #8


Richard Lobb (Lead maintainer)
Tim Hunt: Co-developer from December 2016
Please login to view contributors details and/or to contact them

Comments RSS

Show comments
  • Richard Lobb
    Sat, Jun 18, 2022, 5:59 PM
    Hi Marcelo. The version of CodeRunner in the development branch on github ( is for Moodle 4.0. I'm calling it a release candidate because it hasn't yet been used on a production server but it passes all the tests, runs all our questions in our question bank and has been installed on several low-volume sites. I'll pull it to the official Moodle plugin repository sometime within the next month.
  • feifei meng
    Tue, Aug 16, 2022, 7:45 PM
    Please update "coderunner" and "Adaptive adapted for coderunner" to Moodle 4.0.2 .
  • Jody Paul
    Sun, Aug 21, 2022, 4:31 AM
    Looking forward to being able to use CodeRunner in Moodle 4. (Currently running Moodle 4.0.3) Will check out the developer's forum to see if I can help.
  • Richard Lobb
    Wed, Aug 24, 2022, 8:06 AM
    I've now released the Moodle 4.0 compatible version as a Release Candidate. Sorry about the delay.
  • Vincent MARECHAL
    Thu, Sep 8, 2022, 5:28 AM
    Hello there,
    Then why Moodle allow to install coderunner as compatible with Moodle 4.0 ?
    I installed it, but it doesn't show in activities list...
    Do you plan to repair this issue, please ?
    I downloaded from github page too, but it's the same, it doesn't show in activities list...
    Thanks in advance, regards.
  • Lisa Carpenter
    Wed, Oct 12, 2022, 4:01 AM
    Just wanted to say a quick Thanks to the team ! This is incredible and is going to save us so much manual marking of work.

    Thanks for maintaining this and making it a thing!
  • Amilton Martins
    Sat, May 13, 2023, 1:41 AM
    Hello Authors.
    Is there some ways to run a code runner question outside of a quiz and have your answer scored?
    For example, running a code runner question on H5P?
  • Richard Lobb
    Sat, May 13, 2023, 7:42 AM
    See It lets you embed questions anywhere. They are "scored" in the sense that the code is marked whenever the student clicks Check. And if you install the "Embedded questions progress" member of the set the teacher can monitor progress. But marks aren't entered into the gradebook.
  • яна сикорская
    Sun, May 28, 2023, 11:04 PM
    good afternoon can i use typescript instead of javascript in coderunner
  • Richard Lobb
    Mon, May 29, 2023, 11:40 AM
    Typescript is not available out of the box. But you can implement any language (provided it runs on Linux) with a bit of Python scripting. See However, you are restricted to asking questions that can run in a batch mode, unless you go to a lot of effort hooking into some sort of GUI testing framework.
  • Ravi Shankar Singh
    Fri, Jun 30, 2023, 5:59 PM
    Hi I am not able to install coderunner in moodle version 4.1 so can any one help me out how can I install this plugin?
  • Richard Lobb
    Sat, Jul 1, 2023, 7:17 AM
    @Ravi Shankar Singh: Please specify the series of steps you have taken to install CodeRunner and give details on what goes wrong.
  • Mystic Robo
    Wed, Sep 27, 2023, 6:36 PM
    I installed coderunner on Moodle 4.2 and I even installed the jobe server. I can confirm that the jobe server is running, but whenever I add a new question using coderunner and I try to save my question the "answer" field will diplay "Failed to run tests". Is there a fix for this, should I downgrade my moodle?
  • Richard Lobb
    Thu, Sep 28, 2023, 5:14 AM
    Coderunner V5.2 works fine with Moodle 4.2. You probably have an issue with the Moodle HTTP security settings or a networking problem of some sort. I've posted some things to try on this forum Failed to run tests: Please respond there.
  • Mystic Robo
    Thu, Sep 28, 2023, 11:42 AM
    Thank you @Richarb Lobb, turns out it was because all outgoing curl requests were blocked in the Http security settings. Thank you so much, I'm really grateful to you
1 2 3 4 5
Please login to post comments