Backup and restore

I don't know the details, but Google Moodlerooms on, I think, mass course creation. They have a contribution that does course restore en mass in off hours.

Greetings.

I am a webmaster; our previous moodle server was hacked, and the only backups were course backups. Restoring the data was a one-off task for me, but I thought I would share what I did as it may prove helpful for other people.

USE THESE INSTRUCTIONS AT YOUR OWN RISK.

I am afraid that you will need to be technically inclined, be using a unix-like system (I'm using Mac OS X), be able to run and edit bash and python files, and not be afraid of the command prompt.


In outline, what I did was copy over the latest version of every course, and wrote a script that acted as an administrator of the website to upload (and then delete) the first course in the list, repeatedly.


I set up a prinstine install of the latest weekly build of moodle on my new server. It is Moodle 1.9.3+ (Build: 20081210). The way my restore works may require tweaking with other versions of moodle. (Well, heck, it will require tweaking for your specific site.)

After much ado, my coworker, who has worked with moodle before, created a new course. It was given an id of 2. Then, as an administrator, I can select the course, and in the Administration block, choose files. This takes me to a page with a url of http://moodle-test.my-domain.net/files/index.php?id=2 ; you will need an equivalent of that URL for the script below. I had to find out where the files live on the hard drive; it turned out to be in a folder named "2" in my MoodleData folder. (Called, in full, /IndirectWebStorage/MoodleData/2/ ).

Now, I had to get the latest backup from every course. Daily backups were being created, and for any given course, there were many backups, but they did not all end at the same date. I 'cd'ed into the directory with all the backups, and realised (after some work) that I could make a list of what I wanted like so:

ls -r | rev | uniq -s 19 | rev > list.txt

and copy them as so:

for FILE in ` ls -r | rev | uniq -s 19 | rev ` ; do cp $FILE /IndirectWebStorage/MoodleData/2/ ; done

Basically, it takes the filenames, sorted in reverse alphabetical order (so files of a later date come first), reverses the filename (so the first character is last), strips out duplicates while ignoring the 19 characters of timestamps, and reverses the name back to normal order.

With the files in the course on the new server, I could restore them, manually. I saw that I would go crazy restoring a couple hundred files, so I set out to automate the task.

A useful tool called "twill" came to the rescue. http://twill.idyll.org/ . It is a python program that simulates a user using a website; it will store cookies and submit forms. I've only started using it and really like it.

Here is my twill script. (Don't copy the lines with the ==== in front). I put three hash marks in front of lines you will need to edit

=========== restore.twill ============
run "import os"

echo "Started"

### Use your own domain, moodle admin name, and password
go http://moodle-test.my-domain.net/login/index.php
formvalue login username moodleadmin
formvalue login password "Use_Your_Password"
submit
code 200 # verify that it worked correctly

# Now, go where we can upload files
### Set this to the files page of the course
### to which you copied your backup files.
### You will need to do so again, further on.
go http://moodle-test.my-domain.net/files/index.php?id=2
follow "Restore"

# Display the course name
find backup-(.*).zip
echo Now restoring __match__
### Un-comment this line if you are using OS X and have Growl
### and growlnotify installed. It will then put up a message
### telling you what course is being restored.
#run "os.system('growlnotify -t Restoring Course -m ${__match__}')"

# Yes, we want to restore
formvalue 1 1 "Yes"
submit

# Continue
submit

# Continue, with default settings
submit

# Restore this course now
submit

# ... wait a while

# Continue
submit

# Now the page is the new course

# Go back to the selection page
### Use the same URL for the course files that you used above.
go http://moodle-test.my-domain.net/files/index.php?id=2

# and delete the file
formvalue 1 file1 1 # set the checkbox
formvalue 1 action "delete"
submit

# Confirm deletion
formvalue 1 7 "Yes"
submit
===============

It goes to the page for course id 2, restores the first file in the list (and notifies me that it is doing so using a Mac program called Growl), and then deletes the file.

(Twill is very verbose, and when I tried the -q flag to quiet it down, I did not see the output I wanted -- about which course is being restored. That is why I made it show the message in a little notification window, using Growl. You may wish to find an equivalent, or perhaps output the data to a file and run 'tail -f' the output.)

After you modify it, run it:

twill-sh restore.twill

in the correct directory at a command prompt.


After I got that working, I wanted it to just keep going. Twill can be extended in python, but it wasn't clear to me how to make it loop easily. Initially, I just ran it indefinitely, but when it hit a bug, it stopped going, and Ctrl-C did not kill it. So, I wrote a bash script to run it until it stops.

======= run_it.sh =======
#!/bin/bash

# This script runs my twill script until it hits an error -- hopefully

echo "Starting"
STATUS=$?

while [ $STATUS -eq 0 ]
do
twill-sh restore.twill
STATUS=$?
done
echo "Finished"
========================

This can be run by issuing

bash run_it.sh

at a prompt in the same directory as the twill script.


Well, it was quick and dirty, but my courses (and users!) have been restoring for a couple of hours and I am still sane. Perhaps someone else might find some use with this. Use it at your own risk, of course.