VPL Question

Question types ::: qtype_vplquestion
Maintained by Astor Bizard
VPL Questions are questions that can fit within a Moodle quiz. They are intended to create small to medium coding exercises, based on the Virtual Programming Lab plugin. They are designed to offer a simple interface for students, while keeping the power and versatility of Virtual Programming Labs.
Latest release:
214 sites
174 downloads
18 fans
Current versions available: 1

VPL Question type

Version: 1.2.6 (November 2019)

Authors:

  • Astor Bizard
  • Florent Paccalet (legacy version).

This software is part of the Caseine project.
This software was developped with the support of the following organizations:

  • Université Grenoble Alpes
  • Institut Polytechnique de Grenoble

Introduction

VPL Questions are questions that can fit within a Moodle quiz. They are intended to create small to medium coding exercises, based on the Virtual Programming Lab plugin.
They are designed to offer a simple interface for students, while keeping the power and versatility of Virtual Programming Labs.

For example:
VPL Question example: Hello World

VPL Questions Architecture

A picture is worth a thousand words, so here is how it works (you can also read the thousand words right below the picture!):

VPL Question architecture

A VPL Question is (as its name says) based on a VPL (Virtual Programming Lab).
The execution and evaluation of the question takes place inside that VPL, which allows powerful customizations.
A single VPL can be used for several questions, as long as it is located in the same course.

In order to execute and evaluate student's code in the VPL, the question sends that code (injected within the required file, at the place of the {{ANSWER}} tag) in a new VPL submission. The submission also includes question-specific execution files. The VPL then replaces its own execution files by the ones present in the submission.
This is what allows several questions to be based on the same VPL: as the execution files are question-specific, different execution behaviors (e.g. different test cases) can be achieved on the same VPL. However, this can not be achieved for standard execution scripts.

Create a VPL Question

Step 1: Create a dedicated VPL

The first step to create a VPL Question is to create the VPL it will be using.
To do so, simply create a new VPL inside your course.
It is (strongly) recommended to create it inside a hidden section of the course, and make it available. The goal is to hide it from students, while still allowing them to make submissions to it.

When the VPL is created, there are a few settings you will have to change before it can be used:

  • Edit settings > Submission period: It is recommended to disable the deadline (Due date).
  • Edit settings > Submission restrictions: Set the Maximum number of files to a large enough number, as submissions will include execution files.
  • Edit settings > Common module settings: If you put the VPL inside a hidden section, make sure the Availability is set to Available but not shown on course page.
  • Execution options > Common module settings: Set Run, Evaluate and Automatic grade to Yes.
  • Execution files: Create a file named pre_vpl_run.sh, and paste the following code in it (this is the code that deals with submitted execution files):
for qvplfile in `ls -1`
do
     file=${qvplfile%_qvpl}
     test "$qvplfile" != "$file" && mv "$qvplfile" "$file"
done
  • Required files: Create one (and only one) required file. You can write some basic code in it (like class declaration in java, an empty main function, ...).

You can also edit/add execution files. Please note that:

  • Standard execution scripts will not be editable per-question. Please edit them from the VPL.
  • Other execution files will be editable per-question. However, please create them first in the VPL (you won't be able to create them from a question).

IMPORTANT: Students submissions on this VPL will be discarded by the VPL Question (this behavior can be changed by an administrator in this plugin settings). This is why it is important to create a dedicated VPL (and not to use a VPL where students may want to retrieve their previous submissions).

Step 2: Create one or more VPL Questions

Once a dedicated is created by following the instructions above, you are now ready to create your VPL Questions !
To do so, create a question either from a Quiz or a Question bank, and select VPL Question:
VPL Question type select

Let's breakdown what you can do in the question editing form:

  • General: You fill find here all standard settings for a question, like question name and description.
  • VPL Question template: Here you can select the VPL you created earlier. This will allow you to edit the required file. Please make sure that it includes the {{ANSWER}} tag: this is where the student's code will be injected. Also, please note:
    • Code written here will not be visible by students.
    • The changes you make to the required file will only be effective for the current question. Required file template edition
  • Answer template: Here you can edit what will be pre-filled in the student's answer box. It can be left empty.
  • Teacher correction: Please write here your correction for the question. This is what will be used as the "Correct answer" for Quiz feedback.
    If Validate is checked, the provided answer will be checked against test cases upon question save. This is very useful to check everything is working fine on VPL side, and that both your correction and your tests are correct.
  • Execution files and evaluate settings: This is where you can edit the execution and evaluation behavior of the question. Once again, all modifications done here will only affect the current question.
    • You can edit execution files (most commonly vpl_evaluate.cases). If there is an execution file that you created in the VPL but you do not need in this question, write UNUSED at the beginning of the file: it will be ignored.
      Note: Execution files are only sent to the VPL upon question evaluation. If you need them during execution (Run), please set them as Files to keep when running in the VPL.
    • You can also change the behavior of the Pre-check button (make it act as the Debug button on the VPL, use the execution files above, or use other files). In the last case, you will be able to edit another set of execution files, this time specific to the Pre-check button.
    • You can finally decide whether students need to achieve perfect VPL grade to get the question correct, or use relative VPL grade to get a proportional note on the question.

Known issues

  • Evaluation failures
    It happens that the VPL on which a VPL Question is based fails to evaluate the submission. This failure results in no grade being obtained at the end of that evaluation. VPL Questions are designed to retry several times in that case, however after some retries we have to declare that the grade for this question submission is 0. Please be aware that this can happen even on right answers, so please consider reviewing evaluation results in the case VPL Questions are used in any sort of graded assignment.
  • Test cases hacking
    Test cases, along with other execution files specific to a question, are sent to the VPL as submission files. Even if the VPL is hidden, it has to be available to students so that they can submit to it. This leads to a possibility for students to find access to the VPL, thus to the execution files they submitted. Please be aware of that possibility - what you decide to do about it is up to you.

Sets

This plugin is part of set VPL - Virtual Programming lab.

Screenshots

Screenshot #0
Screenshot #1
Screenshot #2
Screenshot #3
Screenshot #4
Screenshot #5
Screenshot #6

Contributors

Astor Bizard (Lead maintainer)
Please login to view contributors details and/or to contact them

Comments RSS

Show comments
  • Mon, Mar 22, 2021, 3:40 PM
    Hi,
    @Alexandre: this seems to be very related to my problem: a pre-check works, but the final check does not work ("no grade obtained"). The same holds for a validation in the question configuration: this also does not work.

    When I'm using the jail server on the same machine as the moodle server is running, it works! When I am using a dedicated jail server (which only has a self-signed certificated currently), the validation and check does not work! I don't know if it is related to the self-signed certificate (I have checked the option that self-signed certs can be used) or if it is related to the different networks - but this seems to correlate with the problem Alexandre is facing.

    Best regards,
    Michael
  • Mon, Mar 22, 2021, 8:30 PM
    Hi,

    Currently, VPL Questions are using the default port to communicate with the execution server (instead of the secure port if specified in the VPL configuration).
    This issue should be solved in the next patch.
    If this is actually the cause of your problems, that would actually be wonderful! All could be kinda sorted out by itself tongueout

    I will let you know when this is patched!

    Regards,
    Astor
  • Thu, Mar 25, 2021, 7:06 PM
    Hi @Alexandre Meslé and @Michael Munz,

    I released a new version of the plugin, which should solve your problem!
    The VPL Questions now can correctly connect to the execution server via secure websocket, if this is what is specified in the VPL plugin configuration.

    Please let me know if any problem persists!

    Best regards,
    Astor
  • Thu, Mar 25, 2021, 7:09 PM
    Hi everyone,

    This VPL Questions plugin, as of version 1.3.6, is still not compatible with last VPL version 3.4.1.
    I am hoping to solve this issue before the end of the month.

    Sorry for the inconvenience,

    Best regards,
    Astor
  • Thu, Mar 25, 2021, 11:14 PM
    Hi everyone,

    The compatibility issue with VPL 3.4.0+ should be fixed with VPL Questions version 1.3.7.

    Please let me know if you encounter any problem with this new version.

    Best regards,
    Astor
  • Fri, Mar 26, 2021, 3:41 PM
    Dear Astor,
    thank you for the update of the plugin!
    I have good and bad news zwinkernd
    1) Good news: your assumption that our issue is related to the secure web socket is correct: I can confirm that if we do not use secure web sockets or directly activate port 80 on our execution server, it works!
    2) Bad news: I have updated the plugin on our system. But it still does only work if we open port 80 (!) on our execution server AND if we select ws in the plugin configuration, which currently confuses me. I still keep the server url using https and ssl port 443...

    Best regards,
    Michael
  • Fri, Mar 26, 2021, 5:29 PM
    Dear Michael,
    1) Great for the good news!
    2) That indeed is confusing. I have my mod_vpl configuration set to a server with https://, along with the protocol being set to "always use wss". My server has a closed port 80 and it works fine... I believe you did, but I'm just checking: did you test with this configuration?

    I'm convinced we'll finally get this sorted out!

    Best regards,
    Astor
  • Fri, Mar 26, 2021, 7:05 PM
    Dear Astor,
    I again double-checked, but this is unfortunately true.
    My detailed configuration:

    VPL: 3.4.1
    VPL Question: 1.3.7
    JailServer: 2.7.0

    Server URL:
    https://:443/thu_vpl

    An this is what happens:
    With port 80 opened:
    Always use encrypted (wss) websocket protocol --> No grade obtained
    Always use unencrypted (wss) websocket protocol --> Works!

    With port 80 closed:
    Always use encrypted (wss) websocket protocol --> No grade obtained
    Always use unencrypted (wss) websocket protocol --> Timeout!

    Looking forward to next steps! smile
    The certificate of the web server is self-signed, maybe this can cause trouble?

    Best regards,
    Michael
  • Wed, Mar 31, 2021, 10:05 PM
    Hi Michael,

    What does happen when you "Run" your VPL Question?
    I added you to my contacts, if you want to DM me your execution server adress so I can try to reproduce.

    Best regards,
    Astor
  • Fri, May 7, 2021, 3:37 PM
    I am trying to create a VPL activity after some time of not using it.
    I have ensured I have the latest version 3.4.2 (2021033113) and we are running Moodle 3.9.2+ (Build: 20200918).
    I have created the activity and gone to Test Activity > Edit.
    It ALWAYS asks me to create a new file (even if this is not my first time on this page and I have created a new page previously) and once I create a new file (hello.c, hello.py or hello.py3) I cannot type anything?!
    Any ideas why?
  • Fri, May 7, 2021, 5:48 PM
    Hi @Gary Newport,

    This issue seems to be related to the Virtual Programming Lab activity module (https://moodle.org/plugins/mod_vpl), not with VPL Questions.

    Best regards,
    Astor
  • Mon, May 17, 2021, 4:20 PM
    Newbie question here relating to VPL Quiz question setup. In the regular VPL, I provide a test driver main.c, vpl_evaluation.cases that tests the output of a function bar in file foo.c. This is the file that a student submits to complete the programming assignment.

    In VPL Question Template, I write the following:

    unsigned short bar(char *block, int offset) {
    unsigned short p = 0;
    {{ANSWER}}
    return p;
    }

    In Teacher Correction, I provide:
    p = *(unsigned short*)(offset+value);

    In Execution Files, I have vpl_evaluate.cases, main.c, and foo.c. All of these works in the VPL plugin.

    But, I get an "An error occured during question grading (no grade obtained)." Is this because I'm using VPL 3.3.7 and VPL Question 1.3.5. Or, is there something wrong in my interpretation of VPL Question set up? Any help would me much appreciated.

  • Mon, May 17, 2021, 4:54 PM
    Hi @Prasanna Ghali,

    This problem is known to occur if you are using a secure connection with your execution server. This issue is solved with VPL Questions 1.3.6.
    If you are not using a secure connection, you may want to see exactly what happens on the VPL side. In that case, just try to:
    • Save the question with "Validate" checked (to obtain the "An error occured during question grading (no grade obtained)." message).
    • Then, immediately go to your VPL ; you should see a submission with your VPL Question files.
    • Try to evaluate this submission and see what the evaluation result says.

    Let me know if you continue to encounter any issue!

    Best regards,
    Astor
  • Wed, May 19, 2021, 9:29 PM
    Hello Astor,

    Thank you for the response. We upgraded VPL to 3.4.2 and VPL Question to 1.3.7.
    I'm now able to create VPL questions and use them in quizzes and the grades are being correctly computed. However, is the interface supposed to be like writing code on paper but having VPL auto grade the code. The reason I ask is because the Run and Pre-Check buttons don't have any visible effects for students so my worry is that students might lose points for incorrect syntax in a 10 line function without being aware of a missing comma or semi-colon.
    Thanks, Prasanna
  • Thu, May 20, 2021, 2:21 AM
    Hi @Prasanna Ghali,

    I'm glad it improved! However, Run and Pre-Check are in fact supposed to do something, yes big grin
    If these buttons have no effect only for students, it may be because of VPL settings (is your VPL set to allow "run" action? This can be checked in "execution options" on the VPL side).
    If these buttons have no effect for anyone, can you please provide what is written in your browser console when you click them? (it can be opened by pressing F12 for most browsers / OS, maybe Cmd+Opt+C on Safari)

    Astor
1 2 3 4
Please login to post comments