Most authors seem to assume that if they wish to write CodeRunner questions that use a language not directly supported by the Jobe sandbox, they must first modify Jobe to support the new language. That is not the case. A much easier, more convenient and more maintainable approach is to use a Python question type that compiles (if necessary) and runs the student code in a Python subprocess. Indeed, in recent years at the University of Canterbury nearly all new question types have been scripted in Python, even those using languages built into Jobe. For example, the question type we now use for all question in the C programming course is a Python3 question type parses the student's C program and performs lots of checks on things like function length and use of various C constructs before compiling and running the submitted code.
The template code below shows a simple example in which a Python question prototype is used to define a new question type c_via_python that mimics the built-in c_program question type but provides more flexibility. To create the new question type using this template:
You should now find the new question type c_via_python appearing in the Question type dropdown of the author edit form for a new CodeRunner question. This new question type should behave like the built-in c_program question type but is more flexible; for example, it can easily be extended to perform checks on the submitted C code prior to compilation.
The full question prototype for the c_via_python question type is included in the samples folder of the CodeRunner distribution.
""" The template for a question type that compiles and runs a student-submitted
C program.
"""
import subprocess, sys
# Write the student code to a file prog.c
student_answer = """{{ STUDENT_ANSWER | e('py') }}"""
with open("prog.c", "w") as src:
print(student_answer, file=src)
# Compile
{% if QUESTION.parameters.cflags is defined %}
cflags = """{{ QUESTION.parameters.cflags | e('py') }}"""
{% else %}
cflags = "-std=c99 -Wall -Werror"
{% endif %}
return_code = subprocess.call("gcc {0} -o prog prog.c".format(cflags).split())
if return_code != 0:
print("** Compilation failed. Testing aborted **", file=sys.stderr)
# If compile succeeded, run the code. Since this is a per-test template,
# stdin is already set up for the stdin text specified in the test case,
# so we can run the compiled program directly.
if return_code == 0:
try:
output = subprocess.check_output(["./prog"], universal_newlines=True)
print(output)
except subprocess.CalledProcessError as e:
if e.returncode > 0:
# Ignore non-zero positive return codes
if e.output:
print(e.output)
else:
# But negative return codes are signals - abort
if e.output:
print(e.output, file=sys.stderr)
if e.returncode < 0:
print("Task failed with signal", -e.returncode, file=sys.stderr)
print("** Further testing aborted **", file=sys.stderr)