Backup and restore

Hi!

I found myself in the same situation, in my Moodle 2.4.

I performed the zip operation manually and worked ok also for mi. It tooks 11 minutes for zipping a raw 11Gb (the site course).

Thanks for your information!

Jordi

Actually, since there are so many big courses, I find it interesting to automate this task of building manually target zips that, from command line they are built correctly, but that from PHP script doesn't.

I have a solaris system in the server. I have built this script to help me. Hoping it may be of your interest:

<?php

define('CLI_SCRIPT', true);

require_once 'config.php';

ini_set('display_errors', true);
ini_set('error_reporting', E_ALL | E_STRICT);

global $CFG, $DB;

$nonzipped = array();

$modtimecommand = 'perl -MPOSIX -le \'print strftime "%Y%m%d%H%M", localtime((lstat)[9]) for @ARGV\' ';
exec('grep -l error ' . $CFG->dataroot . '/temp/backup/*.log', $nonzipped);

$backupdir = (isset($CFG->backup_auto_destination)) 
    ? $CFG->backup_auto_destination
    : get_config('backup', 'backup_auto_destination');

foreach ($nonzipped as $i => $logfile) {
    $dir = substr($logfile, 0, strpos($logfile, '.'));
    echo '### Processing unfinished backup course directory: ' . $dir . PHP_EOL;
    $xmlfile = $dir . '/moodle_backup.xml';
    $xml = simplexml_load_file($xmlfile);
    if ($xml === false) {
        echo '*** Could not load xml file: ' . $xmlfile . PHP_EOL;
        continue;
    }
    $targetzip = $backupdir . '/' . $xml->information->name;
    if (file_exists($targetzip)) {
        $zipmodtime = filemtime($targetzip);
        if ($zipmodtime === false) {
            $zipmodtime = exec($modtimecommand . $targetzip);
            $dirmodtime = exec($modtimecommand . $dir . '/.');
        } else {
            $dirmodtime = filemtime($dir . '/.');
        }
        if ((int)$zipmodtime > (int)$dirmodtime) {
            echo '*** Skipping. Zip is up-to-date: ' . $targetzip . PHP_EOL;
            continue;
        }
    }
    $command = 'cd ' . $dir . ' && time zip -r ' . $targetzip . ' *';
    echo '===> Running command: ' . $command . PHP_EOL;
    passthru($command);
}

exit(0);

There are some tricks over there:

1. The $modtimecommand is useful to get the modification time from a file when it is so big that the filemtime does not work properly (and returns false).
2. Using the passthru, instead of an exec, we get the standard output from the command out into our php script.
3. I use this script, namely like post-automated-backups.php in the MOODLE HOME like this: time php post-automated-backups.php | tee post-automated-backups.log. This way we get the total spent time for executing all this at once.

Probably you can build your own automated_backup.php script to just run:

1. the admin/cli/automated_backups.php script, and
2. this post-automated-backups.php script, to complete unfinished zips.

Hoping this helps!

Jordi

Follow Up ... using the command line backup.php script ...

from moodlecode/admin/cli/

nohup php backup.php --courseid=86 --destination=/home/backup/ &

nohup means no hang up ... i.e., run the script in the background
Be sure to use the & at the end because that tells the system to
put the command in the background immediately.   This allows one to logout.
OR if your terminal session disconnects (due to time outs for shells), the
process won't be killed until it's completed.

One can see the output of what the backup script would have shown via:

cat nohup.out

== Performing backup... ==
Writing /home/backup/backup-moodle2-course-25-k-12_math-20150320-0748.mbz
Backup completed.

== Performing backup... ==
Writing /home/backup/backup-moodle2-course-86-techframe-20150320-0751.mbz
Backup completed.

'spirit of sharing', Ken