Moodle in English: How to prime Opcache / Fill Opcache with all Moodle php files

How to prime Opcache / Fill Opcache with all Moodle php files

by Carsten Schöffel - Thursday, April 9, 2020, 6:03 PM

We are currently examining how large we should choose 'Opcache.memory_consumption'. One easy way would be to have a script or function that we could call that ensures that after its execution, all Moodle PHP files got called once (e.g. all moodle files are cached by Opcache).

Is there something like that for moodle? If no, is it difficult to build something like that?

My approach is probably pretty silly. But what about a shell script, that finds all .php-files and executes them once with the command line php (excluding the admin folder)? I know that the php-cli tool uses another php.ini file. I am really not sure if this would be the right approach. Maybe it is not so intelligent to call every possible function of moodle.

I am sorry if this has been asked before, but a search for 'fill Opcache' on this forum just takes forever to load.

Thanks in advance,

Carsten

Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Howard Miller - Thursday, April 9, 2020, 6:38 PM

why?

Opcache preload is new in PHP 7.4. All you need to do is to write a script that iterates over all the php files in Moodle. Unfortunately, Moodle is not yet compatible with PHP 7.4 so you're going to need to wait.

I still don't really see the point. The time Moodle takes to build its own cache will far outweigh any advantage you will see by preloading the cache, IMO.

Average of ratings: -

Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Visvanath Ratnaweera - Thursday, April 9, 2020, 7:48 PM

Won't the (preloaded but unused) cache content expire?

Average of ratings: -

Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Carsten Schöffel - Thursday, April 9, 2020, 9:00 PM

Could you clarify what you mean by "expire"? I think there are two meanings for this:
1. There's a time-to-live value for the cached version of a file. If the cached version exceeds that value it will be removed from the cache
2. The original file got updated but the cached version is still serving the old file.

From what I have understood, Opcache just appends all cached versions to the cache memory. It does not matter if that file got updated or hasn't been cached before. If the cache memory is full, the memory gets cleared and anything starts from new.

Average of ratings: -

Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Howard Miller - Thursday, April 9, 2020, 11:01 PM

AFAIK it checks the modified date of the (PHP) file. If it gets changed it is re-cached. There's some .ini settings to control that.

Average of ratings: -

Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Carsten Schöffel - Thursday, April 9, 2020, 8:49 PM

"why?"
1. Because we need to be sure that the value we set in "Opcache.memory_consumption" is sufficient to contain all PHP files that could possibly be called. Same goes for "opcache.max_accelerated_files". We have more than eight instances.
2. I don't know if this is really that important. But if we prime our cache then users will receive faster load times as there won't be any file that isn't already cached.

"The time Moodle takes to build its own cache"
What do you mean by "it's own cache"? Isn't moodles cache the PHP Opcache? I don't quite understand the last sentence.

Nonetheless, here is the little script, that I wrote to prime / preload our caches for one instance:

echo "Priming Cache";

$systemfiles = shell_exec("find /path/to/moodle/system -type f -name '*.php'");
$datafiles = shell_exec("find /path/to/moodle/data -type f -name '*.php'"); //For the lang files
$files = array_merge(array_filter(explode(PHP_EOL, $systemfiles)), array_filter(explode(PHP_EOL, $datafiles)));

foreach ($files as $key => $file) {
  if (opcache_compile_file($file) != true) {
    echo "Could not compile file ".$file;
  }
}

echo "Compiled all files";

Average of ratings: -

Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Howard Miller - Thursday, April 9, 2020, 10:55 PM

You might want to read this - Caching

I did some experimenting ages ago and I didn't find that Opcache made any difference at all to load times. What it did do was slightly reduce server load averages (which isn't a bad thing). From what I recall, it doesn't use much memory either. The default setting was fine. As I say, this was a few years ago and I don't remember the details.

Moodle is only as fast as the slowest thing and there's quite a lot to go at.

Average of ratings: -

Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Carsten Schöffel - Thursday, April 9, 2020, 11:46 PM

Good to know that it probably isn't going to reduce load times. Indeed we want to reduce the cpu load when many users access the site.

Maybe I'll run that script on my development instance again. But for now it seems that a somewhat vanilla moodle instance consumes about 300mb memory if all files in the system and data folder were preloaded. If you count all .php files you also get well above 10 000 .php files, which is the standard setting for Opcache. But if the users don't access any possible site, this probably isn't going to be a problem for now.

I'll report back if we find any groundbreaking discovery.

Average of ratings:Useful (1)

Odg: Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Martin Božič - Saturday, October 31, 2020, 4:18 AM

I guess you haven't had any groundbreaking discovery with the Opcache preloading yet?

Average of ratings: -

Re: Odg: Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Carsten Schöffel - Saturday, October 31, 2020, 5:42 PM

What do you mean exactly? The script above seems to work for most of the files. For some reason it can not compile some files.

In the end it wasn't such a huge issue for us to proceed solving that problem. If you really have to renew the opcache at the rush hour there will be only a slight delay for the first couple of users until their php files have been compiled.

So no. I did not keep working on it.

Average of ratings: -

Odg: Re: Odg: Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Martin Božič - Sunday, November 1, 2020, 3:24 AM

Well I was wondering what were your findings on performance improvement when using preload.

Average of ratings: -

Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Thorsten Bartel - Tuesday, November 3, 2020, 8:06 PM

I know this is a few months old by now, but since the thread got some recent attention, I feel the need to chime in here:
We have (accidentally) experimented with a disabled Opcache on one of your development nodes. It slowed performance very little when load was small, but very significantly once traffic hits hard. If Opcache is enabled, but too small, I see two options:
1) PHP processes have to wait until Opcache can serve them (worst case)
2) PHP processes have to first check Opcache (and miss), then load and compile the script themselves, increasing memory consumption.

Under load, I would predict that a server has significantly reduced capacity without a properly sized Opcache. Additionally to performance hits that may or may not scale with the amount of memory Opcache is lacking.

Cheers

Average of ratings:Useful (1)

Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Tim Hunt - Friday, April 10, 2020, 1:27 AM

To answer the question about how much memory to allocate to OpCache: I have seen 512MB and 10,000 files recommended as the limits for Moodle. That assume your servers have plenty of RAM so 512 is a small fraction. Those amounts are plenty. If you need to, you can probably get away with a bit smaller.

Average of ratings:Useful (3)

Re: How to prime Opcache / Fill Opcache with all Moodle php files

by Ricardo Caiado - Sunday, June 28, 2020, 1:39 AM

Hi,

We use "Admin tools: Opcache management" (https://moodle.org/plugins/tool_opcache)

That way we can see how much memory is needed and use it rationally.

Start with high values. After a couple of days make the necessary adjustment.

Ricardo

Attachment Screenshot_2020-06-27 Opcache management.png

Attachment Screenshot_2020-06-27 Opcache management(1).png

Average of ratings:Useful (1)

Hardware and performance

How to prime Opcache / Fill Opcache with all Moodle php files