How to restore a moodle site without database backup

How to restore a moodle site without database backup

by Mehdi Belhocine -
Number of replies: 23

Hey guys,

Hope you're doing well today.

Im really sad because of my situation today. I've lost 6 months of work, students, courses, lessons, assignements, corrections, forums, etc.

After a bad manipulation, i deleted the moodle database on my host. I already called them and there's no way i can get it back. I have no database backup. I have my moodle folder, moodledata folder, and i had automated course backups.

Here's my question : is it possible to recover my moodle site ? Kind of rebuild the database ?

I want to recover my student accounts, assignements, grades, forums, and only one big important course with all the activities etc.

Thank you guys for your help on this one... sad


Moodle 2.6 / Shared host / Linux

Average of ratings: -
In reply to Mehdi Belhocine

Re: How to restore a moodle site without database backup

by Ken Task -
Picture of Particularly helpful Moodlers

Ouch!  That hurts!

The 'sad' answer is no, can't 'rebuild the database', but all may not be lost cause you say you have course backups.

First, create an archive (zip/tar) of the code directory and most importantly the 'filedir' inside the moodledata directory with what ever tool you have to do that.  If you don't have such a tool, ask the provider to make a zip/tar of the code directory and the 'filedir' inside your moodledata for you AND provide access to those so you can down load them.

The tricky part here is how to extract the backups from the mess of stuff you'll find in 'filedir'.  So the first questions for you: How did you backup the most important course?  By using 'backup' in the course admin menu while in a course?   If you had automated backups, did you leave the default location to Course backup filearea (that was the default)?

If you left the defaults, your backups are somewhere in moodledata/filedir/

Here's an example of where a backup file might be located in moodledata/filedir/:

./d4/1a/d41a96b0b1816f8472297df956086ce10486a675

The file,  in /d4/1a/

now named 'd41a96b0b1816f8472297df956086ce10486a675' is a

Zip archive data, at least v2.0 to extract

That *might* be a backup of a course in Moodle (you could have uploaded zips in your course).

Is your provider willing to run a script inside the filedir so we can get clues like above?

for f in ./*/*/*; do echo "$f"; file -b "$f"; echo "-------"; done > listing.txt

Look through the listing.txt file ...

we are looking for things like this:

./62/53/6253293ad6fc7d651594771e5c24dc31b6cfa0c2
Zip archive data, at least v2.0 to extract

6253293ad6fc7d651594771e5c24dc31b6cfa0c2 *might* be a backup.

cp ./62/53/6253293ad6fc7d651594771e5c24dc31b6cfa0c2 /otherlocaation/maybebackup.zip

cd /otherlocation/

unzip -list maybebackup.zip (list the contents of a .zip)

With this last command, one will see what's in the zip.

If you see this - or something like it:

  Length     Date   Time    Name
 --------    ----   ----    ----
        0  11-26-13 14:36   activities/
        0  11-26-13 14:36   activities/folder_1242/
      137  11-26-13 14:36   activities/folder_1242/filters.xml
     1093  11-26-13 14:36   activities/folder_1242/logs.xml
      399  11-26-13 14:36   activities/folder_1242/folder.xml
      151  11-26-13 14:36   activities/folder_1242/grades.xml
      809  11-26-13 14:36   activities/folder_1242/module.xml
      137  11-26-13 14:36   activities/folder_1242/roles.xml
       61  11-26-13 14:36   activities/folder_1242/comments.xml
      213  11-26-13 14:36   activities/folder_1242/inforef.xml
       67  11-26-13 14:36   activities/folder_1242/completion.xml
       57  11-26-13 14:36   activities/folder_1242/calendar.xml
        0  11-26-13 14:36   activities/url_2802/
      137  11-26-13 14:36   activities/url_2802/filters.xml
     1094  11-26-13 14:36   activities/url_2802/logs.xml
      554  11-26-13 14:36   activities/url_2802/url.xml
      151  11-26-13 14:36   activities/url_2802/grades.xml
      806  11-26-13 14:36   activities/url_2802/module.xml
      137  11-26-13 14:36   activities/url_2802/roles.xml
       61  11-26-13 14:36   activities/url_2802/comments.xml
       59  11-26-13 14:36   activities/url_2802/inforef.xml
       67  11-26-13 14:36   activities/url_2802/completion.xml
       57  11-26-13 14:36   activities/url_2802/calendar.xml
        0  11-26-13 14:36   activities/url_1239/
      137  11-26-13 14:36   activities/url_1239/filters.xml
     2128  11-26-13 14:36   activities/url_1239/logs.xml
      525  11-26-13 14:36   activities/url_1239/url.xml
      151  11-26-13 14:36   activities/url_1239/grades.xml
      806  11-26-13 14:36   activities/url_1239/module.xml
      137  11-26-13 14:36   activities/url_1239/roles.xml
       61  11-26-13 14:36   activities/url_1239/comments.xml
       59  11-26-13 14:36   activities/url_1239/inforef.xml
       67  11-26-13 14:36   activities/url_1239/completion.xml
       57  11-26-13 14:36   activities/url_1239/calendar.xml
        0  11-26-13 14:36   activities/resource_1241/
      137  11-26-13 14:36   activities/resource_1241/filters.xml
      476  11-26-13 14:36   activities/resource_1241/logs.xml
      598  11-26-13 14:36   activities/resource_1241/resource.xml
      151  11-26-13 14:36   activities/resource_1241/grades.xml
      811  11-26-13 14:36   activities/resource_1241/module.xml
      137  11-26-13 14:36   activities/resource_1241/roles.xml
       61  11-26-13 14:36   activities/resource_1241/comments.xml
      170  11-26-13 14:36   activities/resource_1241/inforef.xml
       67  11-26-13 14:36   activities/resource_1241/completion.xml
       57  11-26-13 14:36   activities/resource_1241/calendar.xml
        0  11-26-13 14:36   activities/resource_1240/
      137  11-26-13 14:36   activities/resource_1240/filters.xml
      688  11-26-13 14:36   activities/resource_1240/logs.xml
      670  11-26-13 14:36   activities/resource_1240/resource.xml
      151  11-26-13 14:36   activities/resource_1240/grades.xml
      811  11-26-13 14:36   activities/resource_1240/module.xml
      137  11-26-13 14:36   activities/resource_1240/roles.xml
       61  11-26-13 14:36   activities/resource_1240/comments.xml
      170  11-26-13 14:36   activities/resource_1240/inforef.xml
       67  11-26-13 14:36   activities/resource_1240/completion.xml
       57  11-26-13 14:36   activities/resource_1240/calendar.xml
       57  11-26-13 14:36   badges.xml
       79  11-26-13 14:36   completion.xml
        0  11-26-13 14:36   course/
      137  11-26-13 14:36   course/filters.xml
     1139  11-26-13 14:36   course/course.xml
    23807  11-26-13 14:36   course/logs.xml
        0  11-26-13 14:36   course/blocks/
        0  11-26-13 14:36   course/blocks/configurable_reports_109/
      137  11-26-13 14:36   course/blocks/configurable_reports_109/roles.xml
       61  11-26-13 14:36   course/blocks/configurable_reports_109/comments.xml
       59  11-26-13 14:36   course/blocks/configurable_reports_109/inforef.xml
      716  11-26-13 14:36   course/blocks/configurable_reports_109/block.xml
        0  11-26-13 14:36   course/blocks/rss_client_153/
     2451  11-26-13 14:36   course/blocks/rss_client_153/rss_client.xml
      137  11-26-13 14:36   course/blocks/rss_client_153/roles.xml
       61  11-26-13 14:36   course/blocks/rss_client_153/comments.xml
       59  11-26-13 14:36   course/blocks/rss_client_153/inforef.xml
      773  11-26-13 14:36   course/blocks/rss_client_153/block.xml
        0  11-26-13 14:36   course/blocks/recent_activity_39/
      137  11-26-13 14:36   course/blocks/recent_activity_39/roles.xml
       61  11-26-13 14:36   course/blocks/recent_activity_39/comments.xml
       59  11-26-13 14:36   course/blocks/recent_activity_39/inforef.xml
      469  11-26-13 14:36   course/blocks/recent_activity_39/block.xml
     4330  11-26-13 14:36   course/enrolments.xml
      651  11-26-13 14:36   course/roles.xml
       61  11-26-13 14:36   course/comments.xml
      269  11-26-13 14:36   course/inforef.xml
       57  11-26-13 14:36   course/calendar.xml
        0  11-26-13 14:36   files/
        0  11-26-13 14:36   files/eb/
    23112  11-26-13 14:36   files/eb/eb18361160ca39c9044204416b7fb92203559302
        0  11-26-13 14:36   files/8b/
     8768  11-26-13 14:36   files/8b/8b4feb1b07b63a833124b24ec13d421c82a227ec
        0  11-26-13 14:36   files/3b/
     6994  11-26-13 14:36   files/3b/3b7607c22eba7346820c34321962607c39262e3d
        0  11-26-13 14:36   files/d5/
   323015  11-26-13 14:36   files/d5/d5260a8dee160145f3b2a079441cb6db785b5dc0
        0  11-26-13 14:36   files/fe/
   637767  11-26-13 14:36   files/fe/fe498d3442429a37fd79228236aaf256cb9f865a
        0  11-26-13 14:36   files/77/
    12947  11-26-13 14:36   files/77/77726b8c373455e2dc0844805d6e5f855e41ab24
        0  11-26-13 14:36   files/d0/
     3776  11-26-13 14:36   files/d0/d0e37bd48c19387ce2fae79d3726d8f85a915c9b
        0  11-26-13 14:36   files/bd/
    16051  11-26-13 14:36   files/bd/bd0e7ad0355a24c296886efd208242e2f53d43c3
        0  11-26-13 14:36   files/25/
     3195  11-26-13 14:36   files/25/253bea1bb72945010fdb456d2986c88d29a7796c
        0  11-26-13 14:36   files/af/
     2004  11-26-13 14:36   files/af/af5f3bd5e508843fb8176735896cf44b56ccb142
    11026  11-26-13 14:36   files.xml
     1790  11-26-13 14:36   gradebook.xml
       86  11-26-13 14:36   groups.xml
     7871  11-26-13 14:36   moodle_backup.xml
       83  11-26-13 14:36   outcomes.xml
       83  11-26-13 14:36   questions.xml
      525  11-26-13 14:36   roles.xml
       79  11-26-13 14:36   scales.xml
        0  11-26-13 14:36   sections/
        0  11-26-13 14:36   sections/section_536/
      359  11-26-13 14:36   sections/section_536/section.xml
       59  11-26-13 14:36   sections/section_536/inforef.xml
        0  11-26-13 14:36   sections/section_568/
      451  11-26-13 14:36   sections/section_568/section.xml
       59  11-26-13 14:36   sections/section_568/inforef.xml
        0  11-26-13 14:36   sections/section_537/
      412  11-26-13 14:36   sections/section_537/section.xml
       59  11-26-13 14:36   sections/section_537/inforef.xml
     8834  11-26-13 14:36   users.xml
        0  11-26-13 14:36   moodle_backup.log
 --------                   -------
  1121161                   125 files

*** IT'S A BACKUP COURSE!  AND THAT ZIP FILE CAN BE RESTORE TO A MOODLE!!! ***

Now the next question ... what do you use as a workstation?  Windows/Mac?

Hope it's a Mac, cause you could do what I've shown above on your Mac.

Now to get your site back on line ... *AFTER* backing up what you have and downloading ...

remove the config.php file from the code directory.   You have a backup of it downloaded and can look at it locally.

Empty the moodledata directory.   You have a backup of it downloaded.  Jot down the path to the data directory cause you will be asked to provide the path.

Create a new blank database for your Moodle.  You know the login/password to it.

Now, in a browser, hit your Moodle site and you should be in the install mode.

Once you have the site up and running, we can then concentrate on how to find and extract backups in the local 'filedir' zip/tar ball you downloaded - depends upon what platform you have locally.

*IF* we find them and then you can restore those to your Moodle.

Realize this might be a little much ... if it is, get this info to your provider ... above was done on a Linux server and they should be able to follow the above and get you some .zips. ;)

** That's the least they can do for you since they can't restore your DB!!!  If they can't/won't do the above, I'd be thinking about where to host Moodle else where!

'spirit of sharing'. Ken

Average of ratings: -
In reply to Ken Task

Re: How to restore a moodle site without database backup

by Mehdi Belhocine -

Hi Ken,

Many thanks for getting back in here with a complete and clear solution. I had automated backups enabled in the defaut folder. For me, the really important question is : can i restore all student accounts, their grades, assignments, forums etc following your method ?

If not, it would be much easier for me to create the course from scratch and explain to my students what happend and start everything all over again.

Thanks again for you time.

Average of ratings: -
In reply to Mehdi Belhocine

Re: How to restore a moodle site without database backup

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

Yes, restoring the course backups (as long as you backed up users and user data in the course backups) will restore all grades, enrollments etc. 

If you have a way to search your file structure, you can just search for large files or files with the mbz extension - that is sometimes the quickest way to find them.  I would just rename your moodledata folder and then create a new one, create the new database and then remove your config.php file to start the fresh install.  Once your site is backup then import your course backups.  Depending on how many courses you have, it shouldn't be too painful.  The hardest part will be finding the backup files.

Average of ratings: -
In reply to Emma Richardson

Re: How to restore a moodle site without database backup

by Mehdi Belhocine -

Thank you Emma for this light of hope smile Have a nice day !

Average of ratings: -
In reply to Ken Task

Re: How to restore a moodle site without database backup

by Mehdi Belhocine -

When i have the strange "6253293ad6fc7d651594771e5c24dc31b6cfa0c2" from filedir folder, how can i transform it to a real backup file ? I just change the extension of the file ? If yes, in what ?

Many thx !

Average of ratings: -
In reply to Mehdi Belhocine

Re: How to restore a moodle site without database backup

by Ken Task -
Picture of Particularly helpful Moodlers

Think I'd be safe and copy the '6253293ad6fc7d651594771e5c24dc31b6cfa0c2' file out to another directory AND rename it at the same time:

Make sure check out the contents of that file.  You may not need to, IF, you didn't upload any zips to your course.   But still, you need to see the structure of the zip is in the structure that Moodle can work with.  Important files in that zip/mbz is moodle_backup.xml ... it's the road map for all the rest of the stuff in a course.

cp 6253293ad6fc7d651594771e5c24dc31b6cfa0c2 /someotherlocation/backup1.mbz

forgot to tell you, Moodle 2, recognizes .mbz files are 'moodle backup zips' when restoring.

or if you had turned on the experimental feature of 2.6 for using tar.gz, then to look at them one has to have a program to be able to look into a .tar.gz file.

Emma has already answered that if you did full course backups, the backups will have student info as well as their assignments/grades, etc. *as long as did full course backups*.

However, one cannot search filedir for files named  with .mbz extentions ... they don't exist in filedir.  Now *IF* you had saved the automated backups to another location (I do auto b's this way) then the files in that directory *will* have humanly recognizable names with .mbz extensions.   Don't need to do anything to them, other than download them, and have them ready to use when you get the point in the new Moodle install for restoring courses.
'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: How to restore a moodle site without database backup

by Mehdi Belhocine -

Ok, i got through all folders and subfolders in filedir. I was looking for big files (maybe backups ?). Now im gonna download them one by one on my desktop, rename them in .zip and try to open them looking for that .xml.

Im i right ?

Average of ratings: -
In reply to Mehdi Belhocine

Re: How to restore a moodle site without database backup

by Ken Task -
Picture of Particularly helpful Moodlers

Sort answer, yes, you could try renaming using .mbz and then using something that looks into zips OR .tar.gz files. to see if it is a backup file ... one will see a moodle_backup.xml file IF it is a backup file.

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: How to restore a moodle site without database backup

by Mehdi Belhocine -

Ok, here's what i got so far in terms of file sizes :

  • 1 very big guy at 1Go
  • 1 file at 100Mo
  • 2 files at 95Mo
  • 23 files at 50Mo (corresponding to my 23 videos of my important course)
  • Some files between 1Mo et 4Mo
  • Huge amount of files under 1Mo

I downloaded the 100Mo file and renamed it in .mbz. When a i open the archive, i get an error : the end of the archive is incorrect. However, i can look in it and i see an "activities" folder, and "page_68" subfolder, and finally a roles.xml (impossible to open it because of corrupted archive).

Ps : thx for your time again !

Average of ratings: -
In reply to Mehdi Belhocine

Re: How to restore a moodle site without database backup

by Ken Task -
Picture of Particularly helpful Moodlers

When you downloaded, what did you download AND how did you download?  

Folks on shared hosting usually have to use FTP.  FTP transfers should be done in the 'binary mode' and NOT 'text/ascii mode'.  The only reason I suggest that is the error you get with the one you attempted to extract.  Which, BTW, looks like it might have been a backup ... activities and sub page directories is a good sign.

Are you able to 'look' into one of those content hash named files to see what's contained?  You doing that with what? 

All files have something in them at the very beginning of the file (header) that ID's it.  If you are on Windows, try to open the file with NotePad!   Yeah, I know ... but if one doesn't have a tool to do the job right, then we must improvise.   Now don't get scared when NotePad, churps/beeps and bongs at ya ... it's natural and correct that NotePad with choke on it but it might show you the first line of the file ... and there if it's a zip file you might see PKZIP or a abbreviation of same like PK ... and the rest a bunch of true 'machine langage'.   Just DON'T SAVE THE FILE IN NOTEPAD!!! 

Believe it or not that's a good sign.   At least it's the right 'file type' ... but, maybe ... just whacked on download (remember download/upload using FTP ALWAYS in binary mode).

Got a computer buddy with rep to be a whiz?   Ask for help! ;)

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: How to restore a moodle site without database backup

by Mehdi Belhocine -

I download the files from the server using filezilla. Actually, you got the point with the binary mode ! Now my archives are not corrupted at all smile

The big files were other secondary courses. So i think that ma main course is the 1Go big guy... The issue here is that i cant download it and upload it because of my internet connection speed.

Also, i cant move the file directly on my server because my host propose net2ftp and i can only rename or delete big files.

So here is my question : is there a way i can access the big file in moodledata/filedir from my moodle file manager ?

If its possible, i can rename the file in mbz on the server. Then access it from moodle to restore the backup.

What do you think about that Ken ?

Average of ratings: -
In reply to Mehdi Belhocine

Re: How to restore a moodle site without database backup

by Ken Task -
Picture of Particularly helpful Moodlers

Restoring backups where backup files are on the server is possible but **not** from filedir.   Frist, I think net2ftp has the ability to 'copy, move, delete' ... at least non-restrict versions of the software.

IF that's the case, then here's what I think I'd do:

1. rename current moodledata directory to moodledataold

2. create a new moodledata directory with same owerships/permissions of the old one.

3. Re-install the moodle code by moving the config.php file to another location.  Before re-installing via browser, open config.php and jot down on paper the variables for the following:

 $CFG->dbhost    = '';

$CFG->dbname    = '';

$CFG->dbuser    = ''

$CFG->dbpass    = ''

$CFG->prefix    = '';

$CFG->wwwroot   = ''

$CFG->dataroot  = ''

You should create a blank database called 'moodle' with character set utf8 and collate utf8_general_ci.

Everything in place and double checked, hit the site with a browser.

4. after the moodle has been re-installed and up and running, create another account that has admin level access (don't skip this).   Then go to the new moodledata directory and create a directory called 'repository'.   Inside that directory create another directory called 'xfer'.   Set ownership permissions like that of moodledataold.

5. copy the files you've identified as backups to the new moodledata/repository/xfer/ directory.  Rename the id'd backup files to course1.mbz, course2.mbz whatever.   Check ownership/permisisons.

6. In the Moodle Admin UI, setup a file system repository:

https://docs.moodle.org/27/en/File_system_repository

7. Restore your courses

After you get your site restored and before you begin doing anything else, investigate how you can backup your database (normally done via database 'dump' to an .sql file).  Investigate how you can download (*in binary mode*) that .sql file.

Check with provider that they can/do *backup* your database.   If they cannot/will not, I'd seriously look for another provider.

'spirit of sharing', Ken

Average of ratings: -
In reply to Mehdi Belhocine

Re: How to restore a moodle site without database backup

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

You might try creating a repository in your new moodle site that points to the folder that your backup is in..

Average of ratings: -
In reply to Emma Richardson

Re: How to restore a moodle site without database backup

by Ken Task -
Picture of Particularly helpful Moodlers

Yes, I thought about that. 

Sounds like the path of least resistance, but ....   since moodledataold/filedir is outside of moodledata to be used for the active site and to create a file system repo the folder has to be in moodledata/repository/ that would mean a symlink in /moodledata/respository/ to /pathto/moodledataold/filedir/   which is possible if customer can make symlinks on a shared system.

However, If one were to attempt to backup the active moodledata directory (full site backup),  that backup would be very large ... larger than needed.

Would think that setting up only what is needed the cleaner/better way of doing things at this point and in this situation.

My 2 cents of course! ;)

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: How to restore a moodle site without database backup

by Mehdi Belhocine -

Thank you so much for your help guys !

Here's what i've done before seeing your answers :

  1. Upgraded from moodle 2.6 to 3.0 copying core files to my moodle folder, but i kept the old moodledata.
  2. Used SSH to move big files on the server. So i put my big 1Go backup file in moodledata/repository/backups/
  3. Restored my course from moodle dashboard.

Everything seems to be ok, but i'll only know when students are gonna use the plateform !

Here's my question : is it ok to let things like that or do i have to start all over again by doing what you said to have a cleaner install ?

Ps : if one day you come to paris, please let me know i would love to invite you to dinner !

Average of ratings: -
In reply to Mehdi Belhocine

Re: How to restore a moodle site without database backup

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

You should be fine - the moodledata might be a little confused but if it is not throwing errors, I would leave it for now.  To avoid the pain in the future see if you can set up some kind of backup for the database as well as your moodle and moodledata folders.  Also, backup your courses to a specific folder (this can be done in the automatic backup settings) instead of or as well as the default area.  I have rebuilt a moodle site from course backups many times with no issues.

Glad you were able to resurrect your site!


Average of ratings: -
In reply to Emma Richardson

Re: How to restore a moodle site without database backup

by Mehdi Belhocine -

Thx Emma ! Oh boy thats really the word : a resurrection !

Yes trust me, backing up everything i can is my priority now ! I'm even gonna backing backups LOL smile

Average of ratings: -
In reply to Mehdi Belhocine

Re: How to restore a moodle site without database backup

by Ken Task -
Picture of Particularly helpful Moodlers

Congrats!

I'd keep the old moodledata directory until I was assured student data is present ... this is, of course, unless students didn't have any assignments that they've submitted before/at the database 'disappearance'.   Uhhhh, don't think I said 'start over', did I?  By 'cleaner', I meant not leaving an old filedir IN something that would be backed up.

It's a VERY good idea now, to create a full site backup ... everything (since you've upgraded to 3.x as well) ... the code directory, DB dump, and the data directory.   And establish a 'habit' of a weekly (or sooner) DB dump and archive OFF the server.

Thanks for offer for dinner in Paris!  I wish! ;)

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: How to restore a moodle site without database backup

by Mehdi Belhocine -

So many thx ! You guys rocks !

Ps: im really serious for the dinner smile Please do not be shy to contact me if you come arround here some day !

Take care !

Average of ratings: -