How can I speed up backups on a clustered server environment?

How can I speed up backups on a clustered server environment?

by Mikhail Janowski -
Number of replies: 4
Hi,


I have setup multiple web servers that all have a shared moodledata folder.

I was able to speed up the site using Memcached and $CFG->localcachedir.

However backups are painfully slow because they are created in the shared moodledata/temp/backup folder which is very slow, especially when the course contains many files.

The docs say that $CFG->tempdir can't be a local folder (what's the reason for that?). 

Does anyone know of a way to avoid having to use shared storage to assemble the backup files before they are compressed?

Could I symlink moodledata/temp/backup to a local location or would that break things?


Thanks,
Mikhail

Average of ratings: -
In reply to Mikhail Janowski

Re: How can I speed up backups on a clustered server environment?

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

temp storage has to be shared as - in the general case - intermediate data may be created by a process on one front end and then consumed by a process on another. You can guarantee (presumably) that backups won't do that but there isn't a separate option for backups. 

Unfortunately, I don't have an answer other than to find a way of creating faster shared storage. 

Average of ratings: -
In reply to Howard Miller

Re: How can I speed up backups on a clustered server environment?

by Chris Fryer -

How is moodledata shared? NFS, or Wiindows? Is it feasiblle to put PHP on the server hosting the files and run the backup tak there? That way, moodledata would be local to the backup script. 

You would need to be careful with memory usage. If your file server goes OOM it might choose to kill the NFS daemon, which will knacker your whole site.

Chris

Average of ratings: -
In reply to Chris Fryer

Re: How can I speed up backups on a clustered server environment?

by Chris Fryer -
Just to follow up on my own post... we have a dedicated backup server that has moodledata mounted over NFS.  It has its own config.php in which I've specified a local $CFG->tempdir.  The backup process creates a directory called "backup" in that location, and none of the data is required by the web servers.
Average of ratings: Useful (1)
In reply to Chris Fryer

Re: How can I speed up backups on a clustered server environment?

by Jeff White -

That was actually pretty clever.

It seems rather difficult to request a high performed shared storage location. 

I was experimenting with using a shared ramdisk /tmpfs mount. Seemed to work pretty darn well but was terminated due to a major rule in business IT: Don't do experimental designs production. 

Average of ratings: -