Question types: VPL Question

qtype_vplquestion
Maintained by Picture of Astor BizardAstor Bizard
VPL Questions are questions that can fit within a Moodle quiz. They are intended to create small to medium coding exercises, based on the Virtual Programming Lab plugin. They are designed to offer a simple interface for students, while keeping the power and versatility of Virtual Programming Labs.
132 sites
237 downloads
13 fans
Current versions available: 1

VPL Question type

Version: 1.2.6 (November 2019)

Authors:

  • Astor Bizard
  • Florent Paccalet (previous versions).

This software is part of the Caseine project.
This software was developped with the support of the following organizations:

  • Université Grenoble Alpes
  • Institut Polytechnique de Grenoble

Introduction

VPL Questions are questions that can fit within a Moodle quiz. They are intended to create small to medium coding exercises, based on the Virtual Programming Lab plugin.
They are designed to offer a simple interface for students, while keeping the power and versatility of Virtual Programming Labs.

For example:
VPL Question example: Hello World

VPL Questions Architecture

A picture is worth a thousand words, so here is how it works (you can also read the thousand words right below the picture!):

VPL Question architecture

A VPL Question is (as its name says) based on a VPL (Virtual Programming Lab).
The execution and evaluation of the question takes place inside that VPL, which allows powerful customizations.
A single VPL can be used for several questions, as long as it is located in the same course.

In order to execute and evaluate student's code in the VPL, the question sends that code (injected within the required file, at the place of the {{ANSWER}} tag) in a new VPL submission. The submission also includes question-specific execution files. The VPL then replaces its own execution files by the ones present in the submission.
This is what allows several questions to be based on the same VPL: as the execution files are question-specific, different execution behaviors (e.g. different test cases) can be achieved on the same VPL. However, this can not be achieved for standard execution scripts.

Create a VPL Question

Step 1: Create a dedicated VPL

The first step to create a VPL Question is to create the VPL it will be using.
To do so, simply create a new VPL inside your course.
It is (strongly) recommended to create it inside a hidden section of the course, and make it available. The goal is to hide it from students, while still allowing them to make submissions to it.

When the VPL is created, there are a few settings you will have to change before it can be used:

  • Edit settings > Submission period: It is recommended to disable the deadline (Due date).
  • Edit settings > Submission restrictions: Set the Maximum number of files to a large enough number, as submissions will include execution files.
  • Edit settings > Common module settings: If you put the VPL inside a hidden section, make sure the Availability is set to Available but not shown on course page.
  • Execution options > Common module settings: Set Run, Evaluate and Automatic grade to Yes.
  • Execution files: Create a file named pre_vpl_run.sh, and paste the following code in it (this is the code that deals with submitted execution files):
for qvplfile in `ls -1`
do
     file=${qvplfile%_qvpl}
     test "$qvplfile" != "$file" && mv "$qvplfile" "$file"
done
  • Required files: Create one (and only one) required file. You can write some basic code in it (like class declaration in java, an empty main function, ...).

You can also edit/add execution files. Please note that:

  • Standard execution scripts will not be editable per-question. Please edit them from the VPL.
  • Other execution files will be editable per-question. However, please create them first in the VPL (you won't be able to create them from a question).

IMPORTANT: Students submissions on this VPL will be discarded by the VPL Question (this behavior can be changed by an administrator in this plugin settings). This is why it is important to create a dedicated VPL (and not to use a VPL where students may want to retrieve their previous submissions).

Step 2: Create one or more VPL Questions

Once a dedicated is created by following the instructions above, you are now ready to create your VPL Questions !
To do so, create a question either from a Quiz or a Question bank, and select VPL Question:
VPL Question type select

Let's breakdown what you can do in the question editing form:

  • General: You fill find here all standard settings for a question, like question name and description.
  • VPL Question template: Here you can select the VPL you created earlier. This will allow you to edit the required file. Please make sure that it includes the {{ANSWER}} tag: this is where the student's code will be injected. Also, please note:
    • Code written here will not be visible by students.
    • The changes you make to the required file will only be effective for the current question. Required file template edition
  • Answer template: Here you can edit what will be pre-filled in the student's answer box. It can be left empty.
  • Teacher correction: Please write here your correction for the question. This is what will be used as the "Correct answer" for Quiz feedback.
    If Validate is checked, the provided answer will be checked against test cases upon question save. This is very useful to check everything is working fine on VPL side, and that both your correction and your tests are correct.
  • Execution files and evaluate settings: This is where you can edit the execution and evaluation behavior of the question. Once again, all modifications done here will only affect the current question.
    • You can edit execution files (most commonly vpl_evaluate.cases). If there is an execution file that you created in the VPL but you do not need in this question, write UNUSED at the beginning of the file: it will be ignored.
      Note: Execution files are only sent to the VPL upon question evaluation. If you need them during execution (Run), please set them as Files to keep when running in the VPL.
    • You can also change the behavior of the Pre-check button (make it act as the Debug button on the VPL, use the execution files above, or use other files). In the last case, you will be able to edit another set of execution files, this time specific to the Pre-check button.
    • You can finally decide whether students need to achieve perfect VPL grade to get the question correct, or use relative VPL grade to get a proportional note on the question.

Known issues

  • Evaluation failures
    It happens that the VPL on which a VPL Question is based fails to evaluate the submission. This failure results in no grade being obtained at the end of that evaluation. VPL Questions are designed to retry several times in that case, however after some retries we have to declare that the grade for this question submission is 0. Please be aware that this can happen even on right answers, so please consider reviewing evaluation results in the case VPL Questions are used in any sort of graded assignment.
  • Test cases hacking
    Test cases, along with other execution files specific to a question, are sent to the VPL as submission files. Even if the VPL is hidden, it has to be available to students so that they can submit to it. This leads to a possibility for students to find access to the VPL, thus to the execution files they submitted. Please be aware of that possibility - what you decide to do about it is up to you.

Sets

This plugin is part of set VPL - Virtual Programming lab.

Screenshots

Screenshot #0
Screenshot #1
Screenshot #2
Screenshot #3
Screenshot #4
Screenshot #5
Screenshot #6

Contributors

Picture of Astor Bizard
Astor Bizard (Lead maintainer)
Please login to view contributors details and/or to contact them

Comments RSS

Show comments
  • Picture of Andreas Siebel
    Tue, Jun 16, 2020, 4:50 AM
    Hello,

    I wanted to test VPL+VPL Question with the demo jail server with a simple python "Hello World"-Program. I can run the program but when I click on pre check or check the evaluation throws the error: "Operation aborted by VPL. Execution resources limits may have been exceeded."

    As far i can see, this happens after ~5 seconds.

    This is the test case i'm using:
    #Case = HelloWorld
    output = "Hello World"

    Is this a problem with the demo jail server? (Does it take too long to evaluate test cases?) or could't there be anything which is wrong configured?
  • Picture of Andreas Siebel
    Tue, Jun 16, 2020, 5:40 AM
    After some small bugfixes: When I click on "Precheck", the correct test case-evaluation is shown for some miliseconds - Then the "ressoures exceeded" message is shown.

    I have created a second VPL wiht same settings but without VPL Questions - Here I don't get this error and i can evaluate the VPL with the same jail server without any problems.
  • Picture of Andreas Siebel
    Tue, Jun 16, 2020, 3:20 PM
    I deleted the code lines

    setTimeout(function(){
    if(!this.resultSet){
    callback({execerror: M.str.qtype_vplquestion.execerrordetails});
    }
    },600);

    In vplservice.js. Now it works without problems. Are these lines really necessary?
  • Picture of Astor Bizard
    Tue, Jun 16, 2020, 5:03 PM
    @Andreas Siebel Hello,
    I am aware of this problem, which I corrected. However, it seems that I forgot to upload the new version...
    Thank you for your feedback, it will be fixed soon!
    Note: Yes, these few lines are necessary, when they are correctly written... smile
  • Picture of Andreas Siebel
    Mon, Jun 29, 2020, 4:59 AM
    Hello,

    i have another Issue: When I copy or clone a course with vp_questions, all questions will be invalid, because the id of the VPL activity has changed.

    For me, this would be a roadblock - Is there any idea how this could be resolved?
  • Picture of Astor Bizard
    Tue, Jun 30, 2020, 4:30 PM
    @Andreas Siebel Hello,
    This is for sure the biggest flaw of the VPL Questions. I am aware of it, and finally came up with a fix - this is what the 1.3.0 version is all about.
    You will now be able to clone questions, then simply edit them and change to a new VPL (especially a clone of the old VPL). You then will be prompted to merge contents, which should finalize a proper copy of the question and keep it functional.
    Feel free to update to the newest version and to share any further feedback, it is really helpful!
  • it's me
    Tue, Oct 27, 2020, 5:14 AM
    Dear Astor,

    with some PHP versions, the plugin causes a fatal error in cron due to a method, that does not declare the return value like the parent class does. I published a fix:
    https://github.com/center-for-learning-management/moodle-qtype_vplquestion/commit/695c35f1549b4b1d5e5fe63f2254c42af944b698

    Would be great, if you could add that to the official repo.
    Kind regards,
    David
  • Picture of Astor Bizard
    Tue, Oct 27, 2020, 4:47 PM
    @David Bogner Hello,
    I wasn't aware of such a problem... I find it quite strange that a minor inconsistence like this can trigger a fatal error!
    I will include a fix in the next released version.
    Thank you for your help!
    Best regards,
    Astor
  • Picture of Stephen Parry
    Mon, Nov 2, 2020, 10:22 AM
    Please help.
    I have followed the instructions above to the letter. My VPLs are Python 3. The VPLs themselves work fine, but when I try to use them in a question, even once I have chosen the template vpl and saved, all the text areas such as Edit Template remain empty and do not show line numbers or other code editing features. In preview the run and check buttons do nothing and the console is not shown. If I enable debugging output I can see the following warning:
    Warning: Invalid argument supplied for foreach() in /srv/www/cable.chapeltownacademy.com/html/vle/question/type/vplquestion/locallib.php on line 37
    What is wrong please?
  • Picture of Astor Bizard
    Mon, Nov 2, 2020, 5:20 PM
    @Stephen Parry hello,
    Can you please check that the names of your files do not contain any spaces or special characters?
    And if not, please open your javascript console (F12 on most browsers) and provide a screenshot of its contents.
    Looking forward to sort this out,
    Astor
  • Picture of Stephen Parry
    Tue, Nov 3, 2020, 3:27 PM
    I almost never use spaces in filenames - spawn of the devil ;)
    I have a screenshot that shows some javascript syntax errors, but I cannot paste here?
  • Picture of Stephen Parry
    Wed, Nov 4, 2020, 3:04 AM
    Thanks, have sent.
    I think whatever breaks does so when you pick the template, but I am not sure why. I have tried a Java VPL too - same problem.
  • Picture of Stephen Parry
    Wed, Nov 4, 2020, 10:15 PM
    Hi, Can I just you got that OK?
  • Picture of Astor Bizard
    Wed, Nov 4, 2020, 10:27 PM
    Hi, can I just you got that ok yourself?
    (sorry for the little troll, it's just that I replied to your email - maybe check your spam folder) smile
  • Picture of Henrique Belo
    Wed, Nov 25, 2020, 9:34 AM
    Hello,
    Today I was trying to make some tests on this new question type, I think will be very usefull for my students, but I'm unable to get it working correctrly.
    When I go to test the Pre-check program I'm getting the Execution Error: Not saved: Maximum number of files exceeded.
    What can I do?
    Best Regards
    Henrique
1 2
Please login to post comments