I have a few questions about Moodle. Why are the processes that take a long time (backup/restore/import) built with a php/mysql client/server solution? I see in our network environment that we have problems losing the connection or we get timeouts during these long processes.
Our web and data tier are separated environments and with processes like backups or restores this seems to become a bottleneck. What are your experiences in this matter and how could we improve performance with these long processes?
Backup and restore in Moodle
Number of replies: 12Re: Backup and restore in Moodle
Can you tell us about the environment you have (Moodle version, web server type, database software, operating systems)?
What errors or issues do you see when the operations being attempted fail?
Network operations will have timeouts in order to be able to handle failures, hopefully it's a case of tuning those to get your backup/restore/import operations working as expected.
Re: Backup and restore in Moodle
Leon we use Moodle 3.1.5 on Redhat 7 / Apache 2.4 + PHP 5.6 + MariaDB 10.1
The Moodle webstack is in a DMZ environment and our database/Moodle data is in a business environment.
Local caching on the webstack improved the performance, only our network routing seems to be a bottleneck at this moment with backups and restores/imports. We're trying to find out what's causing the problem. It could be the proxyserver or our own Moodle configuration.
Re: Backup and restore in Moodle
Not sure what '...Moodle data is in a business environment' means technically but assume that means moodledata is either another dedicated internal only server or a file system of some sort on an internal only server.
If one looks at the config-dist.php file one can see some config items that relate to moodledata/temp/
// It is possible to specify different cache and temp directories, use local fast filesystem
// for normal web servers. Server clusters MUST use shared filesystem for cachedir!
// Localcachedir is intended for server clusters, it does not have to be shared by cluster nodes.
// The directories must not be accessible via web.
//
// $CFG->tempdir = '/var/www/moodle/temp'; // Directory MUST BE SHARED by all clsuter nodes.
// $CFG->cachedir = '/var/www/moodle/cache'; // Directory MUST BE SHARED by all cluster nodes, locking required.
// $CFG->localcachedir = '/var/local/cache'; // Intended for local node caching.
There have been some Moodle admins who had moodledata on the web server and most subdirs there ... like temp/backup/ which is used by both backup and restore for building backups or for un-archiving right on the web server.
The only folder/directory that was not really in moodledata on the web server was filedir where moodle actually stores files. That directory was on segment of the internal network dedicated to file storage. The entity made no exceptions to that setup.
Have seen one moodle setup that way and one could tell it was just a little slower when doing something that accessed those file but not really a bottleneck.
'spirit of sharing', Ken
Re: Backup and restore in Moodle
We have a separate webserver and database server.
Data is on the database server (via NFS share).
We don't use clustering and we use memcached for caching on the webserver.
Re: Backup and restore in Moodle
Re: Backup and restore in Moodle
Crontab on the webserver for the cron.php
Re: Backup and restore in Moodle
Saw in the other reply that moodledata is on NFS. Searching for 'NFS' in this forum, https://moodle.org/mod/forum/search.php?search=NFS%20forumid%3A94&id=5&perpage=10&page=1, you may find for youself that NFS doesn't have a good reputation in terms of Moodle performance.
Re: Backup and restore in Moodle
Yes, we use NFS share. We are also going to look for alternatives if it doesn't work out for our Moodle performance.
Any advice on this matter is welcome! Thanks for the link Visvanath, very useful.
Re: Backup and restore in Moodle
Hi Alain,
give https://moodle.org/mod/forum/discuss.php?d=310501#p1242382 a read.
HTH,
Matteo
Re: Backup and restore in Moodle
Thanks Matteo, I'll take a loot at the link. I believe I used it a while ago to tune our NFS settings.
What's your opinion about the 'async' versus 'sync' setting? I tested this and 'async' was a lot faster
with uploading a file to the NFS server (moodledata share).
(I tested with a file of 10GB with dd if=/dev/zero of=/path/to/moodle/nfs/server/tesfile.bin bs=1M count=10000 conv=fsync)
Re: Backup and restore in Moodle
Yeh - we're going to need a proper description of your configuration. Skip the marketing terminology please
I suspect something a bit unusual...
Re: Backup and restore in Moodle
Hi Howard,
What do you suspect a bit unusual? We don't have a very unusual configuration I think..