Moodle optimization

Re: Moodle optimization - answers

by David Delgado -
Number of replies: 10
Thank you for your responses. Martin, I am sorry for not putting my own research regarding each questions before asking for help. That is what I am going to do now (in a simplified way). I would like to know about your opinion on these answers. I would be very pleased if this discussion helps us all to improve our Moodle servers by sharing our knowledge. All I know is by reading the documentation, searching the forums and experimenting with my own test server.

- Which resources (memory, HD, CPU, bandwith,...) are needed for a given number of courses and students?

The HARD DISK is not a big problem: leave about 20MB for the basic installation of Moodle (files and database). Then it depends on the resourses and use of the courses, and the number of students in each one. 2 critical points here: the resources you put in the course and the size of the asignements you ask the students to send. I could say... about 10-20MB per course?. That means, in the worst case, about 9 courses with 200MB ( www.moodle.com basic hosting, that can be upgraded at 10$ per 100MB/month)

The CPU AND MEMORY problems are more problematic, since all Moodle is based on PHP, wich is an interpreted language, that is, each time a page is read, it has to be translated to "machine language", and it takes so much CPU and memory. MySQL is also a big problem, since any page in a course in Moodle uses lots of calls to MySQL tables.

One critical point here is the planning of Quizzes they seem to be the most intensive process in Moodle, and it is better to avoid having lots of students making a quiz at the same time (one of our fellows got his Moodle service suspended by a server overload when making a quiz for just one course).

So, it is better to have as much memory and CPU as you can. Martin says you can start with a 512MB dedicated server. It would be nice to have a dual Xeon MP at 2.8GHz, for example, and 4GB of RAM ( www.moodle.com has a single P4 2.6GHz with 1,5GB for a shared Moodle hosting planned for about 10 Moodle services, wich I guess is quite enough). * By the way, these problems could be fixed a lot if the critical processes of Moodle were implemented in C and loaded as CGI's (much faster)*.

Regarding bandwith, assuming a tipical student with about 200 page hits per month, and about 100K per page, that is about 20MB per student per month. All these means that we can hold about 50 students per GB of bandwith (that means that in www.moodle.com we can host about 500 students, and we can upgrade at about 2$ per GB, that is, per 50 students)

So, for example, with www.moodle.com we can have, with the basic service, about 500 students in 9 courses, having a good server optimization designed just for Moodle (200MB of Hard Disk, 10GB of month transfer). We can find hostings with a much better offer in HD and transfer, BUT, they are not designed for Moodle and its requirements, nor have special Moodle support, and you can find your Moodle service suspended in them.

- What are the features we need to look for in a server to get the best Moodle performance? (PHP Acellerator, PHP Safe Mode = OFF, Control Panel, SSH,...)

Martin says: wink

1. Use LINUX if at all possible (simply because of the amount of support available on the Internet)
2. Get a decent amount of RAM: 512Mb, even 1Gb.
3. Install Moodle using CVS (for easy updating!
4. Switch off unnecessary daemons
5. Tune your Apache configuration to suit usage
6. Automate all your BACKUPS
7. Use redundant disks (eg RAID).
8. Use PHP-Accelerator
www.php-accelerator.co.uk

I add:

9. Make sure to have PHP safe mode = OFF
10. It will help to have CPanel (control panel to manage everything), SSH (secure unix shell access) and Fantastico Moodle 1.0.9 autoinstaller

Try to get all this for your server, or from your hosting. An easy way to get all these (but Fanstastico autoinstaller) is just to go to moodle.com. smile

- Is Fantastico Moodle autoinstaller a good thing to use? (I have tested it and I think that it is a very good starting point, since it assures a smoth install, with no problems, anyway, I think that you have to check and improve the installation afterwards)

The question nearly is self-answered. I tested it, it runned pretty well, BUT, remember that is just an automatic process, designed to suit in every environment. You would better double check everything after it is finished (directory permissions, etc.). Anyway, it works, at least at the server I tested it. The most important thing here is that it helps to avoid the nasty installation problems that many of us have found when installing Moodle "manually", and it is a way to assure a smooth installation.

- Which permissions are the best ones for the different directories?

I think that it depends on if you have root (Administrator) access to the server or not. If you do, the best way is to change the permissions of the moodle directory (eg. /home/moodle/www/moodle) to 0755 (only you can change it, but EVERYONE can read it, including apache) [BY THE WAY, IN THE DEVELOPER WIKI, RED HAT INSTALL, THEY SAY "chown -R apache:apache /usr/moodle", THAT IS, APACHE PROCESS CAN WRITE OR DELETE WHATEVER IT WANT FROM THE MOODLE PROGRAM FILES THEMSELVES :-O , I THINK IT WOULD BE REAL BAD SECURITY PROBLEM!!!]
Change the owner of the moodledata directory(eg. /home/moodle/moodledata) to the web server process, that is, apache process (can be named apache, nobody, etc.) and change moodledata permissions to 0700 (drwx------). So, only the web server (and, of course, the root) could read and modify the files. Remember to change that setting also in config.php, in the moodle directory:
$CFG->directorypermissions = 0700;

If you do not have root access (for example, in a shared server), you can keep the ownership of the moodle directory and change its permissions to 0755 (only you can change it, but EVERYONE can read it, including apache). For the moodledata directory, if you can change the ownership, change the permissions to 0755 and then change the ownership to the apache process. (only apache can write, and EVERYONE can read, so you can make backups, and if you want to delete any file, just do it as admin, entering the course). Remember to change that setting also in config.php, in the moodle directory:
$CFG->directorypermissions = 0755;

If you cannot change ownership of your directories, just change the permissions of your moodledata directory to 0777 (EVERYONE CAN WRITE, POSSIBLE SECURITY PROBLEM HERE). Remember to change that setting also in config.php, in the moodle directory:
$CFG->directorypermissions = 0777;
Any other idea to solve this security hole? sad

- What is the best way to configure moodledata directory? (permissions, owner,...)

Answered above

- What are the best Moodle settings for a server?

1. At config.php, in your moodle directory, be very carefull about the data you put in there. It is the most common cause of the program failure. Doublecheck any data and not put any spaces at the end of the file. Take care about the directorypermissions (read above).

2. At www.yoursite.org/moodle/ , entering as admin, Configure variables, make sure that secureforms = NO, it will help you to avoid lots of problems, specially people using firewalls cannot access your site if it is set to YES. If you do not have the option to upload a picture to your profile, check gdversion, if it is not detected, you have to install it to fix it. If you have problems using files that you have uploaded, if you are using Apache 2, set AcceptPathInfo on in httpd.conf, if not, or if you still have problems, change slasharguments option at Configure variables. If you want to save time to your students and bandwith, it is my oppinion that it its much better to turn htmleditor to OFF (I see no real advantage from using it, but being easier do things that nearly nobody does wink ).

For security reasons, do not use common names for the directories (moodle, moodledata,...) or the databases (moodle, moodleuser) and be very carefull with the passwords (do not use moodle, god, admin, your name or bithday, etc.)

- What could be the best way to implement automated backups?

Wow! In brief: backup your moodle directory, backup your moodledata directory, backup your moodle database. Many ways to do this, I think it is better on a daily basis.

You can take a look at the backup process described in www.moodle.org/doc/ , at "Upgrading".

- What could be the best way to maintain a Moodle installation?

If everything in the installation and configuration is ok, you should not have to spend a lot of time maintaining your service. One good thing to do is staying up to date in this Community. Fix any bug you find, and add any module you like (Chat, for example is good).

Other good things to do if you are the responsible of online training in your organization is:

- Teacher training (online, face 2 face)
- Help desk (including FAQs, manuals, tutorials,...)
- Instructional design (making good courses)

- How can we get rid of the nasty crackers that break our servers?

Some things you can do:
- Fix any bug you can find in Moodle.
- Take care about the directory permissions (see above).
- Take care about the name of the directories and databases.
- Take care about the passwords.
** Make sure that your server is secure **.
- Most important thing: it has to be updated everyday FIXING any bug (there are automatic processes that can do this for you, such as autorpm, or the ones that come with Linux distributions)
- Must have a FIREWALL protecting it. The best way is to have a corporative firewall, plus an internal firewall for the server.
- Must review your LOGS. You can easylly do this with tools such as logwatch.
- Must assure that your critical FILES have not changed. checkpackages can help you with that.

Take a look at:
http://linux.kaybee.org
ftp://ftp.kaybee.org/pub/

- Anything else you can think of

I am exhausted. tongueout Please, contribute to this thread.

I look forward to read your opinions on this.
Average of ratings: Useful (2)
In reply to David Delgado

Re: Moodle optimization - answers

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Wow, David - great job and a lot of work!

It's a very comprehensive summary of the issues - something like that should definitely be in the docs!

I'll try and make sure it gets in for 1.1.
Average of ratings: Useful (1)
In reply to Martin Dougiamas

Re: Moodle optimization - answers

by John Gone -
Impressive, very impressive! What a contribution to the Moodle documentation! Your effort is inspirational. Thanks David.
Average of ratings: Useful (1)
In reply to John Gone

Re: Moodle optimization - answers

by Art Lader -
Wow, thanks! What a wonderful response.
Average of ratings: Useful (1)
In reply to Art Lader

Re: Moodle optimization - answers

by David Delgado -
Thank you very much for your comments. This are just my 2 cents of contribution to Moodle documentation. Anyway, I am sure it can be improved a lot. I hope that someone adds something or fixes anything that could be wrong (I am completely sure that I am not perfect wink ). We are just sharing knowledge, so that "nobody has to invent the wheel again".

I really like this Moodle Community a lot. This was the way WebCT became the most popular e-learning tool in the past years. Now it has become just a money business and many of its past users are moving to other platforms like this (one of them being me). This will never happen to Moodle, as it is a pure GPL, Open Source, Free software ("Free, as in freedom"). smile
Average of ratings: Useful (1)
In reply to David Delgado

Re: Moodle optimization - answers

by Paul Hague -
Thanks for the contribution David,

In response to your question about the security hole when the admin can't change the ownership of the moodledata directory - you could get apache to create this directory using a php script - it would probably have to be 2 levels down from the root directory? I might be able to expand on this if required smile

A note of caution about the php accelerator - we have a bog standard RH 7.3 installation using only orignal RH rpm packages and PHPA caused problems when I initially installed moodle - everything worked fine when I removed it. I'm not saying it's not a good utitliy but it's worth bearing in mind. Maybe install it after you know everything else is working to test it first.

Note: the $CFG->directorypermissions setting shouldn't be effected either way.
In reply to Paul Hague

Re: Moodle optimization - answers

by Paul Hague -
at the command line ...

mkdir moodletest
chmod 777 moodletest/

create script ...

<?php
mkdir ("./moodletest/moodleapache", 0700);
?>

after running the script, at the command line again ...

chmod 755 moodletest/

In this example moodletest/moodleapache would be
you moodledata directory. Obviously this is
inside your documentroot area but I think this
would usually be the case on this kind of hosting
service anyway.
In reply to Paul Hague

Re: Moodle optimization - answers

by Mark Tyers -

regarding your problems with PHP Accelerator.

I installed it on an up to date setup yesterday and noticed big improvements. On the site it says you need to download the version that matches your PHP version. Is that the cause of your headache?

In reply to Mark Tyers

Re: Moodle optimization - answers

by Paul Hague -
I've just tried installing PHPA on my home/test version. My home machine runs Debian Woody with PHP Version 4.1.2 and ionCube PHP Accelerator v1.3.3r2. Again the same type of problem sad

It worked ok for a while then hung. Looking at it in more detail ...

I stop apache so all the processes are killed and delete all the phpa_ cache files in /tmp. Start apache again 5 processes start up all about 3.5MB in size. I try a few other php pages on my replicated site and everything is fine - corresponding number of phpa_ files have been created and all the apache processes are still nice and small.

Try to use moodle and it works for a while - about 40 LARGE phpa_ files have been created and one of the apache processes has jumped to about 14MB in size. Moodle will eventually hang maybe bringing down apache all together.

After I disable PHPA and restart apache everyting works fine. It's a shame though because the site definitely benefits from the accelerator.

So I've created the same problem on a completely different Linux distribution (both very standard and the Debian machine using "stable" packages only). Does no one else have this problem?!?
In reply to Paul Hague

Re: Moodle optimization - answers

by Gunther Dippe -
I checked the recent /tmp/phpa files and the most common size is around 100k but never larger than ca 500k.
I use PHPA with both Linux RH7.1/7.2 and FreeBSD5.0 without any problems.

At least one of the web servers is medium loaded (it doesn't run moodle though). It was also subject to some kind of external "testing" late in July (without my knowledge at that time) and had to serve some 150000 pages within 15 hrs, without any problems.

I don't know if it is of any help but here are my settings for PHPA (close to identical in all servers).

;;;;;;;;;;;;;;;;;;;;;;
; Added php acc 2003-01-20
; PHP Accelerator extension
;;;;;;;;;;;;;;;;;;;;;;
zend_extension="/home/www/cgi-bin/php_accelerator_1.3.3r2.so"
phpa.phpa = on
phpa.cache_dir = /tmp
phpa.ignore_dirs = "/home/www/html/media/"
phpa.cache_file_ttl = 1d
phpa.shm_size = 12
phpa.shm_user = apache
phpa.shm_group = apache
phpa.enable_php_memory_bug_workaround = 0
In reply to Gunther Dippe

Re: Moodle optimization - answers

by Paul Hague -
Success!

But with the TurckSoft MMCache - have been testing it at home and now on our school server with no problems.

You have to compile it for Linux (Windows binaries are available) but this worked fine on Red Hat and Debian (the php devel package needs to be installed though).