Understanding the needs of the VPL jail server

Understanding the needs of the VPL jail server

by Visvanath Ratnaweera -
Number of replies: 2
Picture of Particularly helpful Moodlers Picture of Translators

Dear all

I'm aware of 4.6. Parameters for limiting the resources used by the requested tasks and Re: possible to have more than 1GB in MAXMEMORY (vpl-jail-system.conf). Still, after the observations made in Collecting the VPL usage and displaying in a running graph I would like to understand the parameters better to get the last bit out of our jail server. 

Let us say, the jail server config has:

#MAXTIME set the maximum time for a request in seconds
MAXTIME=1800

# Maximum memory size in bytes
MAXMEMORY=2000 Mb

# Maximum number of process
MAXPROCESSES=500

#SHMSIZE The size of the "/dev/shm" directory he default value is 30% of the system memory
#This option is applicable if using tmpfs file system for the "/dev/shm" directory
SHMSIZE=30%

and the VPL Moodle plug-in set to:

Maximum execution time mod_vpl | maxexetime = 16 minutes

Maximum memory used mod_vpl | maxexememory = 1 GB

Maximum number of processes mod_vpl | maxexeprocesses = 600

Maximum default execution time mod_vpl | defaultexetime = 4 minutes

Maximum default memory used mod_vpl | defaultexememory = 128 MB

Maximum default number of processes mod_vpl | defaultexeprocesses = 200

and the individual VPL activity is set to:

Maximum execution time = Default (4 minutes)

Maximum memory used = Default (128 MB)

Maximum number of processes = empty, Default (200)

Scenario 1. An idling VPL server with plenty of CPU and 20 GB free RAM is hit with burst of 100 jobs in the very same second, each of the size of 100 MB. 

Result: Given that the CPUs manage it, the server will open 100 processes, each consuming 100 MB, giving a total of 10 GB.

  • When the server can process them in less than 4 minutes, all end well.
  • What happens, when they need 10 minutes? Since maxexetime is 16 minutes, will they finish without an incident?
  • What happens, when they need 20 miniutes, or more? Our maxexetime is 16 minutes. They'll be killed?
Scenario 2. An idling VPL server with plenty of CPU and 20 GB free RAM is hit with burst of 400 jobs in the very same second, each of the size of 100 MB. 

Result: Given that the CPUs manage it, the server will try to open 400 processes, each consuming 100 MB, would mean a total of 40 MB. Since there's only 20 GB available, only half the processes will be started. 

  • Those 200 started will run to end if finished before 16 minutes
  • If needed longer, all get killed?

Having started the topic, I see that it is getting "theoretical". If so, then you don't have to torture yourselves. As mentioned in the beginning, the idea is to tune these parameters to get more from the server. We've hit a ceiling as it is. Planning to touch them, but need some understanding on how they react. How was surprised by how little RAM the (our) VPL processes need but the peak CPU bursts they create.

Average of ratings: -
In reply to Visvanath Ratnaweera

Understanding the needs of the VPL jail server

by Juan Carlos Rodríguez-del-Pino -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Dear Visvanath Ratnaweera,

Your questions are very relevant when tuning a VPL jail server. Below, I'll try to clarify how the limits interact and how the scenarios you describe would behave in practice.

How the Limits Work

There are three independent layers of limits, and the effective limit is always the most restrictive one.

1) Activity Level (VPL Instance)

These limits apply to a specific VPL activity.

At this level, you can:

  • Use the default values defined in the plugin configuration, or
  • Set custom values (but only within the plugin-level maximums).

If the plugin configuration is later reduced, the activity values remain stored in Moodle, but they are effectively capped by the new plugin maximum.

So the effective value at activity level is:

min(activity setting, plugin maximum)

2) Plugin Configuration (Moodle Server Level)

These define the maximum values allowed for any VPL activity on this Moodle site:

  • Maxexetime – Maximum wall-clock execution time (not CPU time).
  • Maxexememory – Maximum memory used by all processes running in a task.
  • Maxexeprocesses – Maximum number of processes a task can run simultaneously.
  • etc.

No activity can exceed these values.

These limits are static configuration caps and do not depend on the current machine load.

3) Execution Server Configuration (vpl-jail-system.conf)

These are the limits configured in the jail execution server.

They define the maximum resources that the execution server will allow per task.

Important clarifications:

  • Before accepting a task, the server checks whether the requested memory exceeds the configured limit or whether sufficient real memory is currently available. If either check fails, the task is rejected.
  • Memory availability is checked at runtime every second.
  • The rest of the limits are enforced by the operating system at the task (user) level.
  • From VPL 5.0.0 onward, memory control is also enforced using Linux cgroups.

How Limits Are Enforced

There are two enforcement mechanisms:

OS-level enforcement

  • Maximum number of processes
  • File limits
  • Other ulimit-style restrictions

Execution service enforcement

  • Maximum execution time (wall-clock time, not CPU time)
  • Maximum memory usage (checked at runtime every second; from VPL 5.0.0 also enforced using Linux cgroups)

If a task exceeds a limit:

  1. The jail requests the task to terminate.
  2. If it does not stop, it is forcibly killed.

Important Concept: Per-Task vs. Global Limits

The limits (MAXMEMORY, MAXTIME, etc.) are per task, not global server limits. A task includes both the compilation and execution phases.

There is no predefined global limit on the number of simultaneous tasks.

VPL behaves as if it owns the entire machine.

For this reason, it is not recommended to share the jail server with other services, because heavy VPL load may starve them of resources.

Your Scenarios

Scenario 1

  • 100 jobs × 100 MB
  • 20 GB free RAM
  • Activity memory limit = 128 MB
  • Plugin max memory = 1 GB
  • Jail MAXMEMORY = 2000 MB

Each job requires 100 MB → below the 128 MB activity limit → allowed.

Total theoretical maximum memory usage:

100 × 100 MB = 10 GB

Available RAM = 20 GB → acceptable.

If jobs finish in 4 minutes

They finish normally.

If they need 10 minutes

Whether they finish depends on the effective execution time limit.

Given:

  • Activity default = 4 minutes
  • Plugin maxexetime = 16 minutes
  • Jail MAXTIME = 30 minutes

If the activity is set to “Default (4 minutes)”, then the effective limit is 4 minutes.

So:

  • If execution time limit is 4 minutes → tasks are killed at 4 minutes.
  • If the activity is configured to allow up to 16 minutes → they can run up to 16 minutes.

If they need 20 minutes

They will be killed when reaching the effective execution time limit.

Effective time limit:

min(activity time, plugin maxexetime, MAXTIME)

In your configuration:

  • Activity default = 4 minutes
  • Plugin max = 16 minutes
  • Jail MAXTIME = 30 minutes

Effective limit = 4 minutes (unless the activity time is increased).

Scenario 2

  • 400 jobs × 100 MB
  • 20 GB RAM

Theoretical maximum memory needed:

400 × 100 MB = 40 GB

Available RAM = 20 GB.

What happens?

Important: The jail does not reserve memory.

Instead:

  • Tasks are requested; if sufficient memory is available at that moment, they start.
  • If not, the task is rejected. If no other execution server is available, the user receives a “server not available” message.

In practice:

  • The system will attempt to start as many tasks as possible.
  • Memory pressure will increase.
  • The system may start to reject tasks. It is not guaranteed that exactly 200 will start and 200 will be rejected.

The actual distribution depends on timing, memory fragmentation, and system overhead.

Additional Note on Compilation

If you are using compiled languages, you may need to increase memory limits to allow the compilation phase to succeed.

In a stressed system, memory limit checks may terminate tasks during the compilation stage.

Kind regards, Juan Carlos

Average of ratings: Useful (1)
In reply to Juan Carlos Rodríguez-del-Pino

Understanding the needs of the VPL jail server

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi Juan Carlos

I'm relieved. I thought I won't find the end of the road. Thanks a lot!

In fact, it is simpler than it looks, the same parameters set at three different levels make it look complicated. Will be very useful in our future iterations. Only thing the kind of busy exams like the one https://moodle.org/mod/forum/discuss.php?d=469302#p1896095 are rare and not always informed early enough and even then I can't extract much data from the user side, Moodle, I'm just a non-editing teacher there. Whatever, I'll see.
 
I prefer to keep this thread open, in case I get the chance to come with the results of a similar exercise.
Average of ratings: -