Setting up moodle on LEMP

Setting up moodle on LEMP

by Raunak Kumar -
Number of replies: 9

HI, folks 

I want to set up moodle on different servers having different specifications. I want to install moodle for the range of 100 users-10000users, I have gone through this doc : https://docs.moodle.org/401/en/Performance but I am still unclear of the database configurations that is how much should be the innodb_buffer_pool_size for the mysql 8 or mariadb . 

The following list I made for LEMP server tuneup which is for

1. Nginx 

- Adjusting worker processes and Worker connections 

-  Enabling content encoding : gzip compression of contents ( but this functionality is already enabled with the moodle site) 

-  Changing static content caching duration on nginx 

-  Changing the size of Buffers 

-  Reducing timeouts 

-  Access Logs 

-  Using with x-send file ( does this help?) 


2. DATABASE (mysql 8 or mariadb -10.5) 

-  what should be the appropriate Buffer_pool_size ( note: when referring to official docs, i am getting that it should be 70-80% of the dedicated RAM of db server but I don't want dedicated server for db , I want to install all of them into one server 

-  Innodb_buffer_pool_chunk_size : 

- max_connections 


Average of ratings: -
In reply to Raunak Kumar

Re: Setting up moodle on LEMP

by Ken Task -
Picture of Particularly helpful Moodlers

Don't think anyone could give you appropriate settings because some of those are based upon usage of the site(s).

Multiple Moodle code bases and 1 DB server on the same machine it becomes a balancing act between web services memory (to handle clients) and DB's (think total number of DB's and sites there - regardless of the number of users).

Suggest you install MySQL tuner and right after you get X instances up and running run tuner.  It will have some 'recommendations' for settings.   For those to be accurate, however, sites must have been used.

max connections an example ... default is 150 I think .. that will have to increase but to what is based upon usage.

buffer pools dependent upon data and instances of buffer pools dependent upon the setting for buffer_pools.

If you have to guess, guess high then monitor ... you might find that the optimal setup will be a dedicated server for DB - want it or not! smile

My 2 cents!

'SoS', Ken



Average of ratings: Useful (1)
In reply to Ken Task

Re: Setting up moodle on LEMP

by Brett Dalton -
Picture of Moodle HQ Picture of Particularly helpful Moodlers
@ken you are right on this one, a lot of this will be start with a reasonable number and monitor for issue, adjust and compare. Every config will be subtly different but the guidelines in the doco and most defaults are a good starting point.
Average of ratings: -
In reply to Brett Dalton

Re: Setting up moodle on LEMP

by Ken Task -
Picture of Particularly helpful Moodlers

@Brett - thanks for that.  Many years ago, MySQL distributed with package 3 sample .cnf's - a small, medium, and large - wish they still did that!   Got to the point where I would choose the large config sample as a base.   Then monitored with simple tool, like top, during 'prime time' days (customer delclared those as I wouldn't have a clue - for awhile).  More often than not a tuner recommendation was applied!

Another factor (for Moodles), how site is using blocks or other - 'site design' - which also relates to theme [inline in moodle].   The 'Who's Online' block on the front page when not logged on (they were thinking 'social' and really shouldn't have been) - for reasons of privacy and for server 'santity'/speed in rendering just the front page.

We don't really know OPS setup ... WHM with cPanels can be deceiving if all domains are really allocated to the same server!  Working with an entity right now that complained of slowness ... well, can't squeeze much juice from a 6 Gig server serving multiple domains - a few WordPresses + 2 international Moodles (internaltional does make a diff too!).

So tools like top, tuner, etc. will help ... if monitored and tuner runs made freqently to start - tweaks/balancing act game is kinda fun ... not so much when there isn't enough resources (better to have more than enough, than not enough)!

'SoS', Ken


Average of ratings: -
In reply to Ken Task

Re: Setting up moodle on LEMP

by Raunak Kumar -
Thanks Ken ,
I have installed MySQL tuner before, but what if I am setting up the server for the first time, and also I have to automate the process of setting up the moodle server for different RAMs, vCPUs and memory with different linux distros .
Yes, MySQL tuner is helpful but what if i do not have any users , I am setting the server for the first time . Is there any configurational changes that i need make for nginx/MySQL/php on the basis of RAM, vCPU and memory . I have gone through moodle docs for the performance and I am not getting an adequate explanation on all these and also the moodle docs for on this topic is kinda deprecated . It is too old .
Average of ratings: -
In reply to Raunak Kumar

Re: Setting up moodle on LEMP

by Ken Task -
Picture of Particularly helpful Moodlers

You are looking for a 'cookie cutter' that doesn't exist *unless* all the systems are the same ... period.

All the explanations/docs anywhere will depreciate in short order, given the 'speed' at which servers change (distros, docker or whatever virtualizations, or whatever, whatever).

Would be wonder full if, when moodle sites where registered with Moodle, such tech specs were shared (along with # of users, version of Moodle, standalone vs split services, number of courses - range of courses (meaning highend to lowend), etc., but that's too much to share and would change over time and usage.

You asked about % of memory ... well, duh (no offense), but % memory for an all-in-one 8 Gig server isn't gonna be the same as a 16Gig box ... not out if the shrink wrap and not used to this point.   That is also dependent upon how the web services are setup.   Ok, I've heard that nginx or apache run as a fastcgi or phpfpm are supposed to handle more web clients .... but how many clients.   Not the same for each server when it starts to be used!!!!

Yes, I could give you specs and numbers ... would they work on your servers?   Maybe for a while ... and from what you've described if you copied and pasted my configs to all boxen, some would be overpowered first week it was used!  And the other end of the spectrum, my config were not enough!

Ok, you asked for it so ...

K12 server, VMWare vrtualization, CentOS 7,

with appropriate PHP version + MySQL versioin.

PHP 7.4.33 (cli),  mysql  Ver 14.14 Distrib 5.7.41

 31 Gig RAM,

free -h
              total        used        free      shared  buff/cache   available
Mem:            31G        2.8G         25G         97M        3.0G         28G

some my.cnf clips

innodb_buffer_pool_size=4G
innodb_buffer_pool_instances=4
innodb_log_file_size=384M
max_allowed_packet = 64M
max_connections=451

Tuner run on that server shows (clips):

[OK] Maximum reached memory usage: 4.6G (14.75% of installed RAM)
[!!] Maximum possible memory usage: 32.7G (104.67% of installed RAM)
[!!] Overall possible memory usage with other process exceeded memory

[OK] InnoDB buffer pool / data size: 4.0G/3.0G

But it's running just fine ... for the number of users using.  It's was configured for a time when most of the student population was using moodle but with coming of Google Classroom, elementary, intermediate, and most if not all of junior high have left the moodle and 'Gone Google'.   Only ones left are the high school .... and not all of them!

And that's another bit of info that's misleading ... after servers have been used for years:

6878 users!

243 courses - many are AP and 'team taught' ... one course, all students in a course but 4 or so teachers.  Some are hidden courses with no activity by teacher or students - kept because there is the hope that teachers will come back to the moodle 'one of these days'.

What's peak times for that box?   Depends ... there is some traffic ALL the time and at all hours!   But, in this case, server is used synchronously ... teacher in front of students in class ... where as your's might be async - not face to face .. and therefore you could do with less in config.

Could go on and on ... but will say this ... the first week this box was brought online I monitored *every day* for 2 weeks ... max_connectons setting moved upwards 3 times ... sometimes during prime time.  451 by the way, *is too* much for how it's being used now ... but I left it! smile

Ok, what does all of the above do for you? :|

Again, no magic cookie cutter! smile

Hey., got an idea for ya ... use ChatBot to config a moodle ... and run it.   But then again, if AI can do it, why does your entity need you?

'SoS', Ken

Average of ratings: Useful (2)
In reply to Ken Task

Re: Setting up moodle on LEMP

by Raunak Kumar -

Thanks  Ken, for your generous and so informative reply . The explanation gave me the rough idea of about how to configure my.cnf based on Server specs, I know with the increasing users I have to increase my hardware specs and again thank you so much . WIll not ask for the cookie cutters again .!  😅

Average of ratings: -
In reply to Raunak Kumar

Re: Setting up moodle on LEMP

by Ken Task -
Picture of Particularly helpful Moodlers

Welcome!   Hey, if you come up with a cookie cutter method, please share back!

And if you do jump into a ChatBot thang to get setings based upon specs, share that as well! smile

Yeah, it's work ... but that's good - you are working and justifying your position at your entity at the same time! smile

'SoS', Ken


Average of ratings: -
In reply to Raunak Kumar

Re: Setting up moodle on LEMP

by Brett Dalton -
Picture of Moodle HQ Picture of Particularly helpful Moodlers

Its been a long time since i've optimised MySQL but the Innodb_buffer_pool is a per schema setting, its for the DB server, so regardless of how many moodle installs you have using the sames server (each I assume will have its own schema) set this to 80% of the available memeory, going higher can kill performance due to excessive swaping to disk.  if you are using the same server for a web server or other memory hungry processes you may need to reduce this.  

The chunk size just needs to be equal to or a multiple of the buffer pool size.  unless you have stupid large amounts of memory just set it the same. it only really comes into play when changing the buffer pool size.

With Max connections there is little harm on setting this reasonably high, it will just eat up more memory for no gain.  generally this is a set to something reasonanble and monitor situation, you start seeing long pauses or connection pool errors increase it.  

One last thing, you might want to look at the Innodb_buffer_pool_instances (I think thats the param), this effectively divides up the buffer pool into individual instances which can improve concurrency, but it will limit the size of any individual buffer pool.  so it will kind of depend on the size of the data tables and queries and total memory if this can be done or will show performance gains.

hope that helps. 

Average of ratings: Useful (1)
In reply to Brett Dalton

Re: Setting up moodle on LEMP

by Raunak Kumar -

Thanks  Brett , 

After going through each and every docs I have been told that the  Innodb_buffer_pool  should be 60-70% of the memory but, what if I have installed webserver and php on the same server with the data base and also OS takes up some of the memory, so my question was that what should be the appropriate % that I shoukd give to  Innodb_buffer_pool  from memory with taking OS, Web server and php all of them consuming memory at the same time ?

Average of ratings: -