build mysql bd from hash files ?

build mysql bd from hash files ?

by Dan Lawless -
Number of replies: 10

We recently performed a migration and upgrade on our moodle site but somehow seem to have lost the mysql db.  We have all the hash files from moodledata but are unable to use those files.  Is there any way to perform a reverse db build that will organize the hash files appropriately?  

Thanks in advance for any help.

Dan L

Average of ratings: -
In reply to Dan Lawless

Re: build mysql bd from hash files ?

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

By 'hash files' I assume you mean the the 'moodledata' files area. 

If, as it sounds like, you have lost the complete database then the files stuff represents only one of hundreds of tables that go to make up the Moodle database. If you have genuinely lost your database with no backup then you have lost your site. 

Sorry to be bearer of bad news. 

In reply to Howard Miller

Re: build mysql bd from hash files ?

by Dan Lawless -

Thanks for response Howard.   

You are correct in thinking I am referring to the files in the moodledata folder (completely in tact).  

What happened is that an incorrect moodle folder was backed up (an archieved version rather than the most recent version).  

Is there anything that can be done with the files in moodledata folder to aid rebuilding a new site?

Thanks again.


In reply to Dan Lawless

Re: build mysql bd from hash files ?

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Absolutely nothing.

Those files are only the stuff that was uploaded. Absolutely everything else was in the database. Even if you take the view that you'd like to recover the files you'll never do it. Again, all the information you need was in the database.

It's bad news I'm afraid. 

In reply to Dan Lawless

Re: build mysql bd from hash files ?

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
> Is there anything that can be done with the files in moodledata folder to aid rebuilding a new site?

Since the file names are jumbled, you can not even recognize in what format they are. The Unix command 'file' can tell you though. You can then give the file name the corresponding extension, open it in a program which can handle the format, identify it and give it a meaningful name.

In practice, you gain little, because the originals of the uploaded files are somewhere, in somebody's computer or a backup.
In reply to Dan Lawless

Re: build mysql bd from hash files ?

by Ken Task -
Picture of Particularly helpful Moodlers

Useful depends upon if teachers backed up their courses, if you had automated backups going to 'filearea', etc.

If you can find files that are gzip in the sea of files in moodledata, they might be backups of courses.

From moodledata/filedir/

find ./*/*/ > hashes.txt

hashes.txt file will be large and lines will look like:

./00/00/
./00/00/00007e0b20e39c78158bbc1268309eec06888b0a
./00/00/00000fc42d94251665e3f2ca6511fdaa352080f3
./00/03/
./00/03/0003bcd231a6ee0d78b8df313643930b566b8d12

Create a bash shell script called possibles in moodledata/filedir/

#!/bin/bash
#
rm possibles.txt
touch possibles.txt
for i in `cat ./hashes.txt`
do
    echo "Hash in que:" $i >> possibles.txt;
    file -b $i |grep gzip >> possibles.txt;
done
echo "Done!";

That will create a possibles.txt file ... a large one - and you'll have to look through it to find what could be backups.

In possibles.txt you, hopefully, will  see entries like this:

Hash in que: ./07/4f/074f9c58068de1f24902744625386c577c40663a
gzip compressed data, last modified: Tue Apr  4 06:40:11 2017

And even if you can't see a name, one could copy them out of moodledata/filedir/X/X/X to a holding directory (/possibles/) by names you created ... pbackup1.mbz.

cp ./07/4f/074f9c58068de1f24902744625386c577c40663a /possibles/pbackup1.mbz

Then using tar to view the contents of the .mbz files you've named (may not be backups) looking for moodle_backup.xml.

tar -tvf pbackup1.mbz |grep moodle_backup.xml
-rw-r--r-- 0/0            5558 2018-09-14 08:56 moodle_backup.xml

If last command shows there is a moodle_backup.xml then one could extract just that file to find the original name of the course.

Pain in the arse.   Very time consuming.   But, you have something ... which is better than nothing - IMHO.

'spirit of sharing', Ken


Average of ratings: Useful (1)
In reply to Ken Task

Re: build mysql bd from hash files ?

by Dan Lawless -

Thanks to everyone for their interest and suggestions.  I'll give Ken's idea a go and see what I can find.

Cheers. 

DL 

In reply to Dan Lawless

Re: build mysql bd from hash files ?

by Dan Lawless -

I've managed to recover a good number of courses using some of the info Ken shared.  Here's what I had to do for those who might find themselves in the same boat one day.

First ... I did not have access to the server so I was stuck working only with a backup of files in moodledata\filedir  so I created a standalone server of my laptop to work from. 

Second ... I used treesize to determine large files within moodledata\filedir  -- very tedious but managable.  

As Ken mentioned, I looked for the larger files which might actually be backup files.   

Once I determined the larger files I simply dragged them into a bogus course and restored them ... if the file was actually a course backup file moodle would simply restore the course (no need to change the course extension to zip or mzb this way)  ... if the file was not a backup moodle would reject the file.   

From there I simply backed up the file (course.mbz) and then restore to the live site.  

Thanks again for everyone's help on this. 

Cheers. 

Average of ratings: Useful (1)
In reply to Dan Lawless

Re: build mysql bd from hash files ?

by Ken Task -
Picture of Particularly helpful Moodlers

Congrats!  There is usually more than one way to 'skin a cat'. smile

Now folks have a Windows way as well - even though it's a little 'harder' ... IMHO.

Needless to say ... but will remind ... bet you are working on backup of database on a daily basis, huh? ;)

'spirit of sharing', Ken


In reply to Ken Task

Re: build mysql bd from hash files ?

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Ken

"Extract the Moodle back-up (mbz) files from moodledata and restore them"

Ingenious!
In reply to Visvanath Ratnaweera

Re: build mysql bd from hash files ?

by Ken Task -
Picture of Particularly helpful Moodlers

'Necessity the mother of invention'????   But thanks!

'spirit of sharing', Ken