Recommended opcache.revalidate_freq settings

Recommended opcache.revalidate_freq settings

by David Aylmer -
Number of replies: 11

Hi moodlers!

I have a general question concerning opcache revalidate frequency. From the opcache article in moodle docs http://docs.moodle.org/27/en/OPcache :

opcache.revalidate_freq = 60

From the PHP docs http://www.php.net/manual/en/opcache.configuration.php#ini.opcache.revalidate-freq

How often to check script timestamps for updates, in seconds. 0 will result in OPcache checking for updates on every request.

So, in attempting to achieve a balance between caching PHP code for performance, but not having to restart the webserver each time some PHP code is edited/generated, a recommendation of 60 seconds is given. I'm curious on what the justification for this value is (and why this is needed at all). If I assume that PHP code changes to the webserver are controlled properly, a process of update / run opcache_reset() would suffice. Perhaps resetting the cache for the entire codebase is a bit heavy handed, when by revalidating timestamps after 60 seconds, small code changes can be deployed quickly enough?

The answer might also be in dynamically generated PHP code (Even when the file creation date is greater than the cache timestamp). Perhaps the cache folder in moodledata for example, config.php? This could be accounted for by invalidating the file after it is generated, or by the use of opcache.blacklist_filename.

Does anyone have any thoughts on this? Is anyone aware of the coverage of dynamically generated PHP files in moodle core (and add-on plugins). Is there any other justification for the recommended setting that I haven't thought of?

Cheers,

-David

Average of ratings: -
In reply to David Aylmer

Re: Recommended opcache.revalidate_freq settings

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I'm wondering why you are making frequent code changes to a site with OpCache enabled? If this is, effectively, a development site then what you don't want is loads of caching IMO. 

Average of ratings: -
In reply to Howard Miller

Re: Recommended opcache.revalidate_freq settings

by David Aylmer -

I can't see where I said I was making frequent code changes to a site with opcache enabled?

I agree with your statement on caching being problematic for development sites. However, what I am talking about is the recommendation on moodle.org docs, for production sites, to revalidate cache after 60 seconds. 

Howard, do you use opcache .validate_timestamps (and therefore revalidate_freq) on any of your production sites?

In addition, part of the point of my original post related to dynamically generated PHP scripts. You know - PHP files being created/updated by moodle, and whether they need to be / are being invalidated at this point. If you never validate_timestamps and moodle generates PHP code on the fly, and these files get cached, AND moodle doesn't invalidate this cache on creating the files, then the cached code will differ from the code base.

Average of ratings: -
In reply to David Aylmer

Re: Recommended opcache.revalidate_freq settings

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Are we talking about things like the muc settings (which generates a config file in the data area)? It's a good point - how would you prove that?

Unfortunately, my (unfortunate) experience is that the people who know about these things don't frequent these forums much and it's hard to get a straight answer about caching. Alternatively, nobody knows. 

My production sites are mostly running APC as 5.5 is still a bit too new for most distros. My test box is running opcache with the recommended settings. I must admit, I have had some weird errors where the generated PHP file for caching (the other caching) threw permission error with no obvious reason. 

If I can reliably reproduce that (I think I can), I'll run it again with OpCache disabled and see what happens. 

Average of ratings: Useful (1)
In reply to Howard Miller

Re: Recommended opcache.revalidate_freq settings

by David Aylmer -

Using opcache-gui I typed in moodledata into the file filter.

Seems muc/config.php is the only php file moodledata opcache at the moment 

moodledata/muc/config.php

hits: 37974, memory: 19.219KB last used: 2014-06-09 16:49:56

Just a thought on your permissions issue... the MUC config.php file is created under moodledata. Perhaps your webserver isn't able to execute php files under moodledata? Is that likely?

Interestingly, my moodledata folder has a whole bunch of PHP files present, including language and mdeploy, but none are cached under opcache except for that single MUC config.php file. Suppose that makes sense. If a php file is never run it'll never get cached.

Average of ratings: Useful (1)
In reply to David Aylmer

Re: Recommended opcache.revalidate_freq settings

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

PHP files are read, they don't require the execute bit.

The file in question was actually /path/to/moodledata/cache/core_component.php. Interestingly, it does not appear in the list of cached files. Like you, no files from the moodledata area are cached. Is this because they are 'included' and are therefore simply part of their parent file?

I'm still not sure if this represents a problem. Moodle writes one of these files and it is included by something else but the old, cached version is used. Hmmm....

Average of ratings: -
In reply to Howard Miller

Re: Recommended opcache.revalidate_freq settings

by David Aylmer -

In common parlance I would generally say a source file would be executed whether it is native or interpreted (via a VM for java bytecode / CLR or otherwise - say PHP), but sure, from a file system / chown point of view - what you're saying makes sense. "execute permissions". This isn't actually what I was getting at - but cheers for the knowledge. I'll concede any unix specific permissions knowledge to you, as I don't regularly work on these systems.

Very interesting conclusion! 

The PHP files that appear in the opcache hash table don't seem to have created separate hash table entries for each source file. I wonder if the required/included code is simply concatenated into one gigantic source file, cached into opcode, and then given a single entry in the opcache hash table. That would mean the calling source file would need to be invalidated, rather then the included file. I'll check this theory out when I get a sec.

It's also probable / likely that moodle HQ have already thought all this through and already invalidate the cache of whatever files are generated. Or maybe they thought it was all a bit too complex to track and that is why revalidate_freq is set to 60... as a "catchall".

Were you able to track down your permission error?

Average of ratings: -
In reply to David Aylmer

Re: Recommended opcache.revalidate_freq settings

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

The permission error seems to have gone away by itself - I don't really like that kind of "fix" wink

I hope you're right and Moodle HQ have thought all this through. I'm not quite as optimistic as you big grin

Average of ratings: -
In reply to Howard Miller

Re: Recommended opcache.revalidate_freq settings

by Matteo Scaramuccia -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hi All,
search for opcache_invalidate in the code and you'll find version.php invalidation as well as a generic invalidate_opcode_php_cache() core function to invalidate any file when required e.g. the MUC config when a new config must be saved.

I guess that taking carefully care of opcache invalidation is the reason behind the statement "since Moodle 2.6, it is the only solution officially supported by PHP developers".

HTH,
Matteo

Average of ratings: -
In reply to David Aylmer

Re: Recommended opcache.revalidate_freq settings

by Ryan Smith -

FYI:

The recommended opcache settings came from the opache developers website. Scroll down and you will see their recommended settings:

https://github.com/zendtech/ZendOptimizerPlus/

Average of ratings: Useful (1)
In reply to Ryan Smith

Re: Recommended opcache.revalidate_freq settings

by David Aylmer -

Thanks for finding and supplying that Ryan. Perhaps some settings have been (blindly? / thoughtfully?) copied, but there are certainly some differences.  I'd suggest max_accelerated files should be increased. (See my edit here under max_accelerated_files), I have another forum post about save_comments, I haven't really thought through enable_file_override yet. It seems appropriate enough - but would depend on the filesystem being used.

None of it answers why opcache.validate_timestamps is being used.

The site you found recommends:

opcache.memory_consumption=128
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=4000
opcache.revalidate_freq=60
opcache.fast_shutdown=1
opcache.enable_cli=1

Where as the moodle 27 docs for opcache recommend:

opcache.enable = 1
opcache.memory_consumption = 128
opcache.max_accelerated_files = 4000
opcache.revalidate_freq = 60

; Required for Moodle
opcache.use_cwd = 1
opcache.validate_timestamps = 1
opcache.save_comments = 1
opcache.enable_file_override = 0

; If something does not work in Moodle
;opcache.revalidate_path = 1 ; May fix problems with include paths
;opcache.mmap_base = 0x20000000 ; (Windows only) fix OPcache crashes with event id 487

; Experimental for Moodle 2.6 and later
;opcache.fast_shutdown = 1
;opcache.enable_cli = 1 ; Speeds up CLI cron
;opcache.load_comments = 0 ; May lower memory use, might not be compatible with add-ons and other apps.

Average of ratings: -
In reply to David Aylmer

Re: Recommended opcache.revalidate_freq settings

by David Aylmer -

The other point is the code coverage on a typical site is probably always going to be much lower than than the actual number of code files present, so 4000 (i.e. 7963) is probably going to be OK for the majority of sites.

Average of ratings: Useful (1)