Large Backup and Automatic Backup Failure

Large Backup and Automatic Backup Failure

by Leon Landels -
Number of replies: 5

Version - Moodle 2.8.1 (Build: 20141113)

I’m currently trying to find an effective way of backing up the website information (in particular courses), however I’m having some trouble with backing up larger courses and with the automatic backup option.


1) Backing up individual courses works okay, until I try to backup the larger ‘Geology’ course on the site. I have attached a screenshot to this email displaying the point at which the backup process stops (see attached image). I’ve found that the other (smaller) courses backup okay as a whole, but this particular course requires selecting a few specific sections with each backup (resulting in 3 or 4 backup files for one course).


2) The other issue I came across was with the Automatic Backup. I set this as active and selected the daily backup option over a week ago, however there still doesn’t appear to be any backup files stored yet.


If you can suggest any way I might be able to find a solution to these problems it would be much appreciated.


Attachment Screen Shot 2016-09-06 at 12.03.12.png
Average of ratings: -
In reply to Leon Landels

Re: Large Backup and Automatic Backup Failure

by Jamie Kramer -

For the first issue, I think you are going to need to have a look at PHP and web server logs. It appears to be stopping after only 5 seconds? The other thing to try is temporarily enabling debugging when you run the backup and see if any additional information/debugging comes up.


For the second issue: you have probably already done this but it is worth a mention. Go to Site Admin > Notifications and make sure you don't see a message that "cron is not running normally" or something to that effect. If you do see such a message then your main Moodle cron is not running at all or is not running at intervals that Moodle requires. 

If Moodle cron is running normally, and you are capturing the output of your Moodle cron, then take a look at this cron output. You should be able to search through that output for a string like "backup". This can help you realize whether the automated backup is getting executed or if there were errors along the way causing it to fail completely. 


Jamie

In reply to Jamie Kramer

Re: Large Backup and Automatic Backup Failure

by Leon Landels -

Hi Jamie,


Thanks for your response.


1) The download does continue for a few minutes, but always seems to 'freeze' (stop) just at the very final stage of the backup (i.e. with about 5 seconds remaining). It backs up okay for the smaller courses (80MB or less in size), however it fails during the backup of this particular course (280MB in size).

I tried enabling debugging during an attempted backup but I couldn't see any additional information on the page (should I expect to see information on the screen I was attempting to backup on, or would it appear elsewhere?).


2) I don't see any error messages about the 'cron not running normally' on the Notifications page.


I'm not sure whether I'm capturing the output of the Moodle cron or not. Could you please advise how I would go about locating the string in which I could search for "backup errors"?


Leon

In reply to Leon Landels

Re: Large Backup and Automatic Backup Failure

by C Schipper -

It might be this problem:

https://docs.moodle.org/27/en/Backup_and_restore_FAQ#I_have_a_very_large_course.2C_over_2GB.2C_and_the_backup_process_stops.


We've also had problems with backing up large courses (manually) and the moodle server or our internet network timing out, so you may want to check that too.

In reply to C Schipper

Re: Large Backup and Automatic Backup Failure

by Leon Landels -

Hi Chantal,


Thanks for your message.


I don't know whether its an issue with the file size as the course I'm attempting to backup is only about 280MB. Although the other courses (roughly 80MB or less in size) seem to backup okay.


It may be that the network is timing out, because I've noticed that the browser loading bar stops in conjunction with the backup processes freezing.


Leon

In reply to Leon Landels

Re: Large Backup and Automatic Backup Failure

by Ken Task -
Picture of Particularly helpful Moodlers

Causes for this behavior could be multiple ...

Things to check:   PHP settings for max times for script to run, memory a script can consume, and extra memory to use (backups will use extra memory already ... hit's hard coded into the backup scripts).   All those can or should be seen in the Site Admin - Server - PHPInfo screen.

From what you've described (and pic of progress) it could be that the course backup has reached a size (physical size - not processing 'size' (there's a difference) OR the combination of both - physical as well as processing (as evidenced by your description of breaking apart the backup into 4).

When a backup gets to the very end the very last process in the whole routine is to copy the backup that uses moodledata/temp/backup/[longdirectoryofnumbersandletters]/ as a build area to the destination to archive.   The destination could be (more than likely is the first): file area [which is moodledata/filedir/ in the new file system somehere) or a designated directory (which system admin has to create manually).    In either case, the routine uses  a 'copy' command to accomplish.

If running a RedHat family based OS, there is a known limitation in the operating system with *copy* command being used for files 4G* ... am assuming, of course, your server is a true server x86x64 (64bit).

If this is the case, upgrading Moodle, BTW, will not fix.

So ... use whatever you have to inspect/browse files and look into:

moodledata/temp/backup/[thatlongletternumberdir]/ that was being used to build the backup.

Is there a backup.mbz file present?   If there is, then it's  a *copy* issue.

Also look at moodledata/temp/backup/ for a .log file (which is ascii/txt) that has the same filename as the backup directory being used to build a backup.   That log file has some info in it ... providing stages of the plan to back the course up ... they are provided as numbers.    A successful backup will reach a 1000 and completes.   A failed backup might have a reference to the last successful stage - less than 1000.   What is that reference? (might give us a clue).

Inspection of the course ... one of the heaviest processing (therefore 'large') modules is quiz.   How many quizzes (test/assessments) are there in the course?

If the course will not backup manually via GUI, then it will certainly fail when part of autobackups.

There is something to try that gets beyond limitations/issues ... in admin/cli/ of the code there are command line scripts that one could run from ssh session.   Two of interest ... backup.php - which takes preferences for backups from the Moodle Admin UI.   It is for a single course.

And there is automated_backups.php - obvious what it does.

The backup.php script takes your Apache web server restrictions/config limitations out of the loop in a backup.  Takes two parmeters ... course ID to backup and destination.

I have a 2.8 sandbox and a script in it's admin/cli/ that does this:

php backup.php --courseid=$1 --destination=/var/www/unifsrepo/m28/

The $1 is a variable and I can pass to my script the course ID to backup.   In your case, provide the command the course ID of your troubled course.    I suggest using the destination option whose directories will not be created by the script so you'll have to create them.

That keeps the backup out of moodledata/filedir/ ... can find it ... can see it ... can inspect it's contents from the command line.

I'll stop at this point cause autobackups (while seemingly related) has some special considerations of it's own.    Would suggest turning off autobackups for now, until this single course backup issue is resolved - no way to exclude this troubled course from autobackups and it could be the culprit in autobackups failing.

'spirit of sharing', Ken