Test case from text file

Test case from text file

by Ron Viner -
Number of replies: 5

Hello,

I am testing students of Algorithms course. since the grade is equivalent to the run-time of their program, the tests given are very large text files, therefore I prefer to keep the input texts as execution files instead of copying the whole text into the evaluate_cases section.

I tried using this guide which you recommended:  http://www.science.smith.edu/dftwiki/index.php/Tutorial:_Moodle_VPL_--_A_Multi-File_Java_Program_with_Data_File

However, even when I click the keep files when running on my text files the VPL dosen't seem to accept this kind of argument as in the guide.

for example when I type

case=Test 1

input= input.txt

output="1"

The VPL interprets the "input.txt"(which is a file I kept in the execution files) as a string to enter the student program.

I would appreciate your help of how to run a test case with a text file of my own.

Thanks alot!  

Average of ratings: -
In reply to Ron Viner

Re: Test case from text file

by Juan Carlos Rodríguez-del-Pino -
Picture of Particularly helpful Moodlers Picture of Plugin developers
Hello. Ron Viner.

I guess that you need (required) to use the standard input to give the data to the student's program.

You can resolve this need with this approach:
1) Create a  script "inputFile.sh" to pass the file to use as standard input to the student's program.

#!/bin/bash
read filename
if [  ! -f "$filename" ] ; then
    echo "Error: file $filename not found"
else
    # Runs student's program with $filename as standard input
    ./vpl_test <$filename
fi

Remember to mark "inputFile.sh" as "file to keep when running". To have the script available at testing time.

2) You must add to each test case at vpl_evaluate.cases file the following:
program to run = ./inputFile.sh


Example:

Case = Test file input
Program to run = ./inputFile.sh
input = data1.txt
output = "Expeted result"


Important: VPL is not ready to evaluate based on the code runtime. VPL does not use a queued execution leading to the possibility that some tasks run in the same execution server at the same time. To palliate this problem the teacher, after all, can reevaluate all the student submissions using a dedicated jail server. In this situation, the evaluation tasks will be run one by one.

Best regards,
Juan Carlos.
Average of ratings: -
In reply to Juan Carlos Rodríguez-del-Pino

תשובה ל: Re: Test case from text file

by Ron Viner -
Dear Juan,
Thanks alot, it seems to solve the problem.
except one little thing, now after I have this test case option available the students program prints with the output this warning :
>/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)

to be a little more specific, the evaluation window contains this :

Incorrect program output
--- Input ---
> input0.txt

--- Program output ---
>/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
>756

--- Expected output (exact text)---
>756

I would appreciate your help with getting rid of this warning in the output of the students program.

Thanks in advance,
Ron
Average of ratings: -
In reply to Ron Viner

Re: תשובה ל: Re: Test case from text file

by Juan Carlos Rodríguez-del-Pino -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Stranger error. I can not reproduce the problem in regular conditions.

I guess it was there before. Please, tell me if when you run (not evaluate) the student's programs the message appears. You can try adding to the script "export VPL_LANG=he_IL.UTF-8" but I am not sure it will work.

Also, It seems that VPL does not manage correctly the conversión from Moodle "he" language to Linux "he_IL.UTF-8" locale.

Anyway, you can work around the problem by installing the "en_US.utf8" locale in your jail servers. Notice that VPL uses this language as a fallback in case of locale setting fails.

Best regards,

Juan Carlos.


Average of ratings: -
In reply to Juan Carlos Rodríguez-del-Pino

תשובה ל: Re: תשובה ל: Re: Test case from text file

by Ron Viner -
It is actually a warning which always appears when running or evaluating, and usualy I just ignore it, just that in this case where the input is taken from a text file the program prints the warning as well as the program output.
I tried adding to the script "export VPL_LANG=he_IL.UTF-8" but unfortunately it didn't help.
I will try to contact the technical department of my university and figure out if it is possible to install the "en_US.utf8" locale.

Thanks and best regards,
Ron
Average of ratings: -
In reply to Ron Viner

Re: תשובה ל: Re: תשובה ל: Re: Test case from text file

by Juan Carlos Rodríguez-del-Pino -
Picture of Particularly helpful Moodlers Picture of Plugin developers

A temporal workaround can be to remove the first line of the student's output.

./vpl_test <$filename 2> > (tail -n +2)

This code will remove the first line of the stderr and do nothing with the stdout. Note that spaces matters.


Best regards,

Juan Carlos.

Average of ratings: -