load balancer with haproxy and memcached
2 backends ubuntu 14.04 PHP5+ Apache
Another one alone for the tests:
ubuntu 16.04 + php7-fpm
DB: mysql 5.7, plenty of RAM
NFS for moodledata
For many years, I've noticed that the avatars of users on the welcome page were displaying slowly, using pluginfile.php. It can be sometimes 17 sec! Today, I spent hours to understand. I put the profiling in config.php and noticed that SQL was fine 8/0 request in max 2ms. So I believe it was due to NFS and cache. I used xdebug and kcachegrind and discovered I was spending 99.3% of the time in session_start
As we use memcached It tried to change the lock parameters without success. Even reading the php memcached driver in C to understand the values. At the end I change moodle memcached.php:
and change that:
ini_set('memcached.sess_locking', '0'); // Locking is required!
Obsviously things were going faster.. Just a few ms for all the icons. Before it was randomly an icon locked for a few sec..
I then looked at session locking to see if they were freed as fast as possible and saw many \core\session\manager::write_close(); // Unlock session during file serving. in filelib.php which is called by pluginfile.php
I don't understand why it is not running well.. Somewhere, there should be a piece of code locking sessions for too long. I hesitate to get rid of session locking in production..