General plugins

New Virtual Programming Lab (VPL) module

 
 
Picture of Antonio Lepore
Re: New Virtual Programming Lab (VPL) module
 
Thanks for your answer but i have another question. Can I made a quiz with authomatic evaluation? And if I can, how?
F.e.
I give  text to students, they make their own program and then system give me an authomatic evaluation of their test.
Thank you!
 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: New Virtual Programming Lab (VPL) module
Group DevelopersGroup Documentation writersGroup Particularly helpful Moodlers

For automated grading of computer programs, https://github.com/trampgeek/CodeRunner is another Moodle add-on that you might want to look at. (It is a question type.)

 
Average of ratings: -
Picture of Juan Carlos Rodríguez-del-Pino
Re: New Virtual Programming Lab (VPL) module
 

Hi Antonio,
You can find more information about automatic evaluation in VPL at the following links:

http://vpl.dis.ulpgc.es/index.php/en/documentation/76-general-documentation
Section "Creating automated tests"
and
http://vpl.dis.ulpgc.es/index.php/en/documentation/58-automatic-evaluation-of-students-submissions

Best regards

 
Average of ratings: -
Picture of Dominique Thiebaut
Having problem creating user vpl_evaluate.sh script
 

Dear Ruan Carlos,

First, thanks for putting together such a useful and sophisticated module for Moodle.

I teach computer science and am interested in setting VPL up for my class next semester.  

I have setup two virtual servers, one running LAMP+Moodle+VPL module, the other one running the vpl-jail-system.

It took me a while, but everything seems to run fine.   I have a simply python 3 program called test2.py that I am currently using as test:

x = float( input( "> " ) )
y = float( input( "> " ) ) 
print( min( x, y ) ) 

And I have a vpl_evaluate.cases file to test it:

case = Test 2a
input = 3
5
output = 3.0
case = Test 2b
input = 3
3
output = 3.0
case = Test 2c
input = 1000
-1000
output = -1000.0


When I run or evaluate the script with the default vpl_run.sh and vpl_evaluate.sh, i.e. without specifying them, I get the console, and I can enter 2 numbers and get the smallest one back, and the evaluate gives me a "0/3 tests fail" which is good.

When I create my own vpl_run.sh script, it also works fine and runs the program in the console.

#! /bin/bash
echo "#! /bin/bash"     > vpl_execution
echo python3.4 test2.py          >> vpl_execution
chmod a+x vpl_execution
However, trying to set the vpl_evaluate.sh script to the code above makes the system hang.
A 1-line window opens up and starts showing a counter showing seconds elapsing: "Evaluating: 3 seg", then 4 seg, 5 seg, etc.

When I look at the vpl-jail-server, I see that a new directory is created

jail/home/p15913# ls -l
total 16
-rwx------ 1 15913 15913 1559 Jun  7 21:28 common_script.sh
-rw------- 1 15913 15913   77 Jun  7 21:28 test2.py
-rwx------ 1 15913 15913  360 Jun  7 21:28 vpl_environment.sh
-rwxr-xr-x 1 15913 15913   32 Jun  7 21:28 vpl_execution
but it seems that vpl_evaluate.sh or vpl_evaluate.cases does not get created there.

I've been tweaking many different things, but I'm not finding out what the problem is.  It looks like my vpl_evaluate.sh is missing something.

Is there something obvious I'm missing?

Many thanks for any info/feedback you could share.

Dominique




 
Average of ratings: -
Picture of Juan Carlos Rodríguez-del-Pino
Re: Having problem creating user vpl_evaluate.sh script
 

Hello Dominique,
Thank for the clear exposition of your problem.
All types of executions (run/debug/evaluate) take two stages: The first generates vpl_execution or vpl_wexecution, the second executes vpl_?execution. But before start the second, the system remove all the execution files (vpl_evaluate.sh, vpl_evaluate.cases, etc.) that you set in  "ADVANCED SETTINGS  ► EXECUTION FILES".
Your 'ls' may be showing that you are in the second stage.
You can go to " ADVANCED SETTINGS /  FILES TO KEEP WHEN RUNNING" and set the files that you miss as to be kept. Also  you can rename the files in vpl_evaluation.sh and the system will not be able to delete them.

Best regards


 
Average of ratings: -
Picture of Dominique Thiebaut
Re: Having problem creating user vpl_evaluate.sh script
 

Thanks for the quick response, Juan Carlos.

Unfortunately, setting the files to keep to all the vpl_run/evaluate/cases does not solve my problem.

However, I discovered that I get an error on the jail server when I attempt to use my own vpl_evaluate.sh:

ERROR: apport (pid 8308) Tue Jun 10 10:16:51 2014: Unhandled exception:
Traceback (most recent call last):
  File "/usr/share/apport/apport", line 298, in <module>
    cwd = os.readlink('/proc/' + pid + '/cwd')
FileNotFoundError: [Errno 2] No such file or directory: '/proc/8307/cwd'
ERROR: apport (pid 8308) Tue Jun 10 10:16:51 2014: pid: 8308, uid: 0, gid: 0, euid: 0, egid: 0
ERROR: apport (pid 8308) Tue Jun 10 10:16:51 2014: environment: environ({})
ERROR: apport (pid 8310) Tue Jun 10 10:16:51 2014: another apport instance is already running, aborting
Apport is a tool that apparently gathers crash information, and I imagine the vpl_jail system is making use of it.   Here it looks like I have two such instances running at the same time.
Any suggestion for fixing this?
Thanks again,

Dominique
 
Average of ratings: -
Picture of Juan Carlos Rodríguez-del-Pino
Re: Having problem creating user vpl_evaluate.sh script
 

Hi Dominique,
the error you show is not due to VPL and I don't know the reason of the problem. Nevertheless, I found that apport is logging a normal action in vpl-jail-server as an error. This is my fault I wrote a call to abort() instead of exit(). This problem (the logging) of apport will be solved in vpl-jail-server 2.0.2.

Returning to your question, I think that the better way to help you to resolve your problem with vpl_evaluation.sh is that you show the details of the activity you are developing.
A backup file with the activity migth be great! if you are using "based on" don't forget to also add this activity

Best regards.
Juan Carlos

 
Average of ratings: -
Picture of Dominique Thiebaut
Re: Having problem creating user vpl_evaluate.sh script
 

Sure, here's a description of the activity:

Test Python 1

Due date: Sunday, 31 August 2014, 8:00 PM
Requested files: test1.py (Download)
Type of work: Individual work
Grade settings: Maximum grade: 100
Run: Yes Evaluate: Yes
Automatic grade: Yes

This is the second trial...

Execution files

vpl_run.sh

    1 #! /bin/bash
    2 echo "#! /bin/bash" > vpl_execution
    3 echo "python3.4 test1.py" >> vpl_execution
    4 chmod a+x vpl_execution
    5 

vpl_debug.sh

vpl_evaluate.sh

    1 #! /bin/bash
    2 echo "#! /bin/bash" > vpl_execution
    3 echo "python3.4 test1.py" >> vpl_execution
    4 chmod a+x vpl_execution
    5 

vpl_evaluate.cases

    1 Case = Test 1
    2 output = "Hello World!
    3 "
    4 



When I test the activity in the VPL administration module, I go to Edit and enter this simple code:

print( "Hello World!" )

which I save.  I click on Run and get the correct output in the console.  

Now, for some reason I don't understand, when I click on Evaluate, I do not get the counter shown seconds elapsing (I haven't changed anything in my setup), but instead I get the output of the program but no evaluation of its output and no grade.


Moodle VPL Evaluate Window


I am forging ahead and learning how to setup VPL activities without having a custom vpl_evaluate.sh file, but I imagine that I will need to do so in the future, so any help on this would be much appreciated.

By the way, I am documenting my progress and putting sample VPL activities on my wiki, mostly so that I can remember how to set this up when the time comes next semester.  Others  may find them useful too:  http://cs.smith.edu/dftwiki/index.php/Tutorials


Thanks again for the superb tool and support!

Dominique

 
Average of ratings: -
Picture of Dominique Thiebaut
Re: Having problem creating user vpl_evaluate.sh script
 

Ah!  I did find what I think is the problem.   I just created a new VPL activity and created my own vpl_evaluate.sh, which is now stuck running and counting seconds as I originally described in my first post.

When I look on the jail server, I see that the files tested are located in /jail/home/p19526:

root@VPLServer:/jail/home/p19526# ls -l
total 32
-rwx------ 1 19526 19526 1559 Jun 11 10:23 common_script.sh
-rw------- 1 19526 19526  233 Jun 11 10:23 gremlin.py
-rw------- 1 19526 19526   97 Jun 11 10:23 sample.txt
-rwx------ 1 19526 19526  408 Jun 11 10:23 vpl_environment.sh
-rw------- 1 19526 19526  175 Jun 11 10:23 vpl_evaluate.cases
-rwx------ 1 19526 19526  162 Jun 11 10:23 vpl_evaluate.sh
-rwxr-xr-x 1 19526 19526   49 Jun 11 10:23 vpl_execution
-rwx------ 1 19526 19526  162 Jun 11 10:23 vpl_run.sh

However, when I check the vpl-related processes running, I see this:

root@VPLServer:/jail/home/p19526# ps auxw | grep vpl
root      2242  0.0  0.0  23520  1000 ?        S    Jun04   5:01 /etc/vpl/vpl-jail-server
root     22910  0.6  0.0  32232  1856 ?        S    10:23   0:00 /etc/vpl/vpl-jail-server
19526    22911  0.0  0.0      0     0 ?        Zs   10:23   0:00 [vpl_evaluate.sh] <defunct>
19526    22913  0.0  0.0  17960  1436 pts/3    Ss+  10:23   0:00 /bin/bash /home/p19526/vpl_execution
root     22915  1.0  0.0  23912  1724 ?        S    10:23   0:01 /etc/vpl/vpl-jail-server
root     22963  0.0  0.0  11744   924 pts/0    S+   10:25   0:00 grep --color=auto vpl

Note that the vpl_execution file was executed from "/home/p19526/", which does not exist.  It should have been "/jail/home/p19526"...

I will try to figure out where/how this setting can be modified...

Dominique


 
Average of ratings: -
Picture of Juan Carlos Rodríguez-del-Pino
Re: Having problem creating user vpl_evaluate.sh script
 

The ps output is normal. Due to chroot, the path

/home/p19526/vpl_execution

is correct.


Best regards


 
Average of ratings:Useful (1)
Picture of Afzaal Shabbir
Re: Having problem creating user vpl_evaluate.sh script
 

Hi Juan,

i have a totally unrelated question, is it possible to install Moodle+VPL PLugin + Jail service install in one virtual machine?

i need to give this logic to one of our client who is unable to understand why we need to have two machines dedicated to moodle.

 
Average of ratings: -
Picture of Juan Carlos Rodríguez-del-Pino
Re: Having problem creating user vpl_evaluate.sh script
 

Yes, it's possible, but not recommended. The load of execution in the jail server can interfere in the execution of Moodle. The security is another problem that can arise.

If you have enough hardware assigned to the moodle server you can install Virtualbox in your virtual machine (yes, a virtual machine into a virtual machine) and install there the jail server.

You must be aware of three things:

1) Your jail server (service) must have assigned two ports differents than defaults (80 and 443) for example 52000 and 52001. Change properly the configuration file.

2) Your jail server ports must be accessible from internet (not only from your Moodle server). You can use NAT port forwarding if using in VirtualBox

3) Use the correct URL (eg http://servername:port/path) in the general plugin configuration in the Moodle server


Best regards

Juan Carlos

 
Average of ratings: -
Picture of Afzaal Shabbir
Re: Having problem creating user vpl_evaluate.sh script
 

Thank you very much for detailed response. really appreciate your help.

Cheers,

A

 
Average of ratings: -
Picture of Juan Carlos Rodríguez-del-Pino
Re: Having problem creating user vpl_evaluate.sh script
 

Hi Dominique,
Thanks for your tutorial, any help is appreciated.

Again your clear report of the issue makes me to point out the problem easily.

The problem is that vpl_evaluation.sh is the script to generate the program/script that evaluates the student's program and NOT the student's program that is going to be evaluated.

The default vpl_evaluate.sh uses the result of vpl_run.sh (vpl_execute) as the program to be evaluated.
Notice that the output of the execution of vpl_execute, generated by vpl_evaluate.sh, is the result of the evaluation. You can see the format of the output of this  execution in the documentation .

Best regards

 
Average of ratings:Useful (1)
Picture of Dominique Thiebaut
Question: How to make output of program evaluated become the grade
 

Thank you so much for all the help, Juan Carlos.

I have another question, if you don't mind.

I would like to have students submit assembly language programs, and one of the requirements for one of the tests would be to generate code that as small an executable as possible.  In other words they would have to pay close attention to the size of their code and of their data.

I have figured out how to test/evaluate an assembly language program.  That's fine. (By the way, I documented the VPL activity here for those interested: http://cs.smith.edu/dftwiki/index.php/Tutorial_Moodle_VPL_Assembly_Language_with_Output)

Now, I know I could add a new program to the "Execution Files" section of the VPL administration block that could get the size of the executable version of the student assembly program.   This second program could easily output 100, 90, 80, 70, etc... depending on whether the size of the student program is within 10%, 20%, 30%, 40%, etc.,  of the best size. 

I would like the output of this second program (with will be a number between 100 and 0) to become the grade assigned to the student program.

I'm still very confused about how to write a vpl_evaluate.sh script that would take this number output by the program and make it the grade.

Any suggestion or hint?

Merci!

Dominique

 
Average of ratings: -
Picture of Juan Carlos Rodríguez-del-Pino
Re: Question: How to make output of program evaluated become the grade
 

In this case I think that an example will be better. Copy and paste the following script into vpl_evaluate.sh and push evaluate in the test activity.
-------------------------------------------

#!/bin/bash
cat >vpl_execution <<END_SCRIPT
#!/bin/bash
# This is an example of evaluation output
#
# One line comment
echo "Comment :=>>This is a one line comment"
# Multiline comment
#start tag
echo "<|--"
echo "This is a multi line comment (line 1)"
echo "This is a multi line comment (line 2)"
echo "This is a multi line comment (line 3)"
echo "This is a multi line comment (line 4)"
#end tag
echo "--|>"
#This is a grade set
echo "Grade :=>>1Value"
# Comments may have some format
echo "<|--"
echo "-Titles start lines with '-'"
echo "Normal text"
echo ">preformated text"
echo "--|>"

#Last grade is taken
echo "Grade :=>>Value"
END_SCRIPT
chmod +x vpl_execution

--------------------------------

Best regards

 
Average of ratings: -
Picture of Dominique Thiebaut
Re: Question: How to make output of program evaluated become the grade
 

Once again, thanks for the quick answer.  This was what I needed to figure out how to make it work.  I can now test a program and assign a grade inversely proportional to the size of the executable version of the program.  

I have documented it here for those interested: http://cs.smith.edu/dftwiki/index.php/Tutorial:_Moodle_VPL_--_Testing_the_Size_of_A_Program%27s_Executable

Thanks much!

Dominique

 
Average of ratings: -