Automated course backups run time estimate?

Automated course backups run time estimate?

by Paul Lindgreen -
Number of replies: 5
Picture of Particularly helpful Moodlers

2-3 years ago automated backups were attempted but they ended up taking too long and killing performance so we decided not to run them. Since then we have decreased course sizes substantially and improved overall performance so I was thinking about having another kick at the can,

Can anyone give me a ball park estimate as to whether the following automated backups should complete within a 6hr time frame without killing performance during that time (ie roughly <45sec per course backup)? We would like to run it 1am-7am when usage is light (<100users at any given time) and before it gets busy at 830am.

500 courses
4500 students
1-50mb per course backup size
moodle 2.7.8
Server: windows 2008R2 64bit, 4 cpu, ram 14gb, php 5.5, using opcache
separate server for Mysql 5.5 , same server specs

If the backup occasionally spilled over into our normal work day hours and killed performance can I manually kill the process as a failsafe?

We are doing this to help teachers who occasionally accidentally delete course activities. We have tape backups but they are too much work to restore for such a small mistake.

thanks.


Average of ratings: -
In reply to Paul Lindgreen

Re: Automated course backups run time estimate?

by Stuart Mealor -

Hi Paul.

These days the Moodle backup system is more sophisticated.

So once you have a single full backup (maybe starting on a Friday night?) you can do things such as:

  • Skip courses not modified for a specific number of days
  • Skip courses not modified since the last backup
So, if you had 500 courses, but only 100 or so had actually changed overnight, you could get to a position where only the courses that actually changed, and therefore need backing up, are processed.  Course that don't need to be backed up are skipped.  So in this scenario, your backup process would take 80% less time !

We use Linux for all our Moodle servers, but I expect the Windows server will be slower.

Average of ratings: -
In reply to Stuart Mealor

Re: Automated course backups run time estimate?

by Paul Lindgreen -
Picture of Particularly helpful Moodlers

I was aware of the skipping of non modified courses and forgot to mention we have about 1000 additional hidden courses in addition to the 500 active courses, we keep the previous years worth of courses on hand for quick access. Of the 500 active courses there may be 100-400 used daily. My moodledata folder is about 260gb, I'm guessing 1/3 or less of that is actively used.

I guess there's only one way to estimate how long this may take, try it out on our development server (much slower than production) and if it is mildly successful try it one night in production and monitor results.

I was wondering if anyone had some real life examples for automated backup duration times. Does anyone think 500 courses in 6hr in the middle of the night is not realistic? If it takes longer than 6hr would that be a red flag of something wrong with our system? Would it be more realistic to budget for 1min/ea course backup rather than 45sec (most course backups <50mb), maybe budgeting for 8hr is safer?

Average of ratings: -
In reply to Stuart Mealor

Re: Automated course backups run time estimate?

by Paul Lindgreen -
Picture of Particularly helpful Moodlers

re: So, if you had 500 courses, but only 100 or so had actually changed overnight

If students just viewed a course would that be a 'change' (course logs change)? Or would they need to update/add/delete something before the course being considered for a course backup?

Average of ratings: -
In reply to Paul Lindgreen

Re: Automated course backups run time estimate?

by Stuart Mealor -

My gut instinct says that someone just viewing a course would not be considered a 'change'

I understand what you are saying about a view of a course being logged, but the logs are central, and not part of the course content (the log tables are just related to courses via the course id).

So I'd be 99% sure a view will not be taken as a change to a course for backup purposes.  Worth a quick test though smile

Average of ratings: -