Performancee statistics

Performancee statistics

by Juan Daniel Burró Aláez -
Number of replies: 4

I would like to use this thread to show our current configuration and test statistics. I would like to get feedback so I could test other combinations and improve as much as possible our servers.

We're going to have many Moodles running so this environment will be replicated as fine tuned as possible.

Currently we have to servers, one for database and the other for Moodle.

Specs are like this:

 Intel(R) Xeon(R) E-2236 CPU @ 3.40GHz 12 processors

64 GBytes RAM

2 hard disks (RAID 1)

In one machine we have two moodle instances running (3.7.6 with Moove theme), both of them dockerized and with https. One moodle instance  is running with apache-mpm prefork (one container) and the other one with nginx and php-fpm (2 containers). The other machine is a MariaDB 10.4 (bare metal).

We haven't tune anything. First issue for us was to decide which technology use for php.

We've used locust for our tests (https://locust.io/). We've tested creating 10/users per second, each user that login makes random requests  (2 to 10 seconds each).

See our stats. Response times show two lines (green line is the average, red line is time to get 95% of requests).

500 users Apache - mpm-prefork:




500 users with nginx-fpm:




What we've found:

Apache and prefork is faster. We didn't expect that. If we load 600 users, it's even more clear. However we've found 2 things that we don't understand:

- Memory usage: In both cases is pretty low. However CPU is really high. See for example this htop with 600 users (apache mod_php), we don't even use 2GBytes of RAM:



- CPU and memory usage is even lower using nginx-fpm, however performance is worse.

As we're new in Moodle environment and also php world any hint would be really welcome. We have just use default configuration (not even APC Cache).  We expect to make those improvements later on, but with this measurements we should stay with prefork.

Average of ratings: -
In reply to Juan Daniel Burró Aláez

Re: Performancee statistics

by Juan Daniel Burró Aláez -
I've just change php-fpm communication between containers from tcp to unix socket but performance is not better. I can get up to 75-80 request per second with prefork, much better than using fpm.
I guess RAM usage is smaller than we expected due to our Moodle being just installed and without much data.
Average of ratings: Useful (2)
In reply to Juan Daniel Burró Aláez

Re: Performancee statistics

by Juan Daniel Burró Aláez -
In case anyone may find it useful we finally decided to go with nginx and fpm.
We did miss XSendFile with nginx that give us a big  increase in performance. Also we found 
pm.max_children = 5 too small, so we increased it to 20.
However we've found a bottleneck in network communication between Moodle and our DB that can be seen as peaks in our graphic (increase in request time). It's just 100Mb/s and we will increase it to 1GB.


We also tested with 1500 users and we get aproximately 150 req/s (Moodle pages) with average request times of 1 sec.  Activating APC as App cache (we use just one frontend) and Redis for session cache we improve request times for just 10%, however the machine feels more stable (cpu). 
Average of ratings: Useful (2)
In reply to Juan Daniel Burró Aláez

Re: Performancee statistics

by Juan Luis Maestre -
Hi Juan Daniel, we've in our production environment nginx + php-fpm7.2 giving us good results. 7 x (4 vcpus + 8 GB RAM). In our case we don't use docker

In load tests a few weeks ago we can manage a lot of users doing quizzes (around 6k) with 5 minutes of ramp up:
http_req_duration: avg=25.64ms min=2.17ms med=13.84ms max=2.16s p(90)=38.65ms p(95)=85.15ms

requests/minute

For us, the key is to have a backend layer as optimized and tuned as possible. Memcached, opcache, ... and of course, a powerful database with a good connection to the frontends

We are currently testing ARM-based frontends with very good results (about 20% performance improvement vs. x86-64)
Average of ratings: Useful (3)
In reply to Juan Luis Maestre

Re: Performancee statistics

by Juan Daniel Burró Aláez -
Thank you Juan Luis. We improved our testing as we had some network delays than distort our graph (response time). Now we get like 200 views / second that it's more or less 200views * 30req/views = 6000 req/second, really far away from your statistics! What I don't understand is that we don't use almost frontend RAM (2-3GBytes) but our frontend CPUS are 100% full. Database is ok.
Average of ratings: -