restore 1.9 backups in 2.2 including user data

restore 1.9 backups in 2.2 including user data

by Klaus Steitz -
Number of replies: 19
Picture of Plugin developers Picture of Testers

Does anybody know, by when it will be possible to restore user data of 1.9 backups in Moodle 2.x?

The 2.1 Release Notes said "No user data (like forum posts, grades, submissions, ...) are supported yet. Blocks are not restored yet."

So like many others i hoped, 2.2 would bring this feature. But apparently it isn´t implemented again. It has been shifted to the roadmap, but only as "other" (http://docs.moodle.org/dev/Roadmap). 

I found a discussion in the tracker (http://tracker.moodle.org/browse/MDL-30160), Martin saying maybe in 2.3. How likely is it that it will come with 2.3 (in 1st June) finally?

Is there a possibility to vote for that development or at least to support it (not by programming it myself)?

Thanks for your answers!

 

kind regards from germany
Klaus

Average of ratings: -
In reply to Klaus Steitz

Re: restore 1.9 backups in 2.2 including user data

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Hi Klaus, yes, this is a disappointment, I understood that v2.2 was going to resolve this issue, but obviously it is not quite ready. AFAIK, the main issue is around meeting a considerably more rigorous security regime than was used in previous versions. My argument though is if you can restore courses, and you can bulk upload your users, then converting to v2.2 is not going to be too disruptive IF it is done at the end of the academic year. I do not think that enrollments should continue between 2 academic years in courses, perhaps in their studies sure, but not Moodle courses. I understand that my employer is going to be updating to v2.2 over the next few weeks, the end of our academic year. Many Moodles in the US are doing so in their summer break, July 2012.    

Average of ratings: Useful (2)
In reply to Colin Fraser

Re: restore 1.9 backups in 2.2 including user data

by Stephen Martin -

That's well and good for those in an academic environment with things like semester breaks and academic terms, but quite a few of us are in corporate or commerical training markets where we don't have the luxury of hard breaks.

Average of ratings: -
In reply to Stephen Martin

Re: restore 1.9 backups in 2.2 including user data

by Klaus Steitz -
Picture of Plugin developers Picture of Testers

thanks for your remarks!
There are courses and training stuff that want user data to be imported from 1.9.

Looks like we have to swallow the pill and find other solutions for them. Admins can´t shift the upgrade to 2.x perpetually, the support for 1.9 will end in June 2012 ("Bug fixes for serious security issues in 1.9.x will continue until June 2012", source: http://docs.moodle.org/dev/Releases). So admins are forced to upgrade in 2012 somehow...

Average of ratings: -
In reply to Stephen Martin

Re: restore 1.9 backups in 2.2 including user data

by Colin Fraser -
Picture of Documentation writers Picture of Testers

Yes Stephen, you are quite right, I do forget that a lot of organizations using Moodle are corporate or commercial based. Please accept my apologies, and this suggestion:

The transition from Moodle 1.9 to v2.x cannot be accomplished easily when you have no opportuntity to make the kind of clean breaks discussed earlier. This then makes it essential to apply some planning to the process. Whatever is done, I suggest you set up a v2.2 and make sure you have all your courses in it before you do anything else.  

It can be done quickly, that is, select a date in the future, prepare a Moodle 2.x and parallel it with your existing v1.9.x, that is, make sure it has all the same courses and users all set up and on that date, everyone moves over on one day.

Alternatively, once you have a v2.x setup, start all new enrollees in it while existing enrollees complete their current courses in the v1.9.x. 

The drawback in the first instance is that it is possible that people will lose their work, their grades from gradebook, their history, which is not really acceptable. It means you have to make sure you have these things properly recorded and available to them if needed. The drawback in the second instance is that you have to make sure that your tutors, teachers, facilitators, mentors etc., can work in both Moodles and that you have sufficient server space to make it all work. 

This can be a starting point, if you have not already considered it.

Of course, there is another method for making the transition while live, well almost. First, create a test Moodle of v1.9.x exactly the same as your production site, complete with user files and everything. Use the upgrade process to trial doing it (you can find more information about upgrading to v2.1 here). Repeat it several times if needed. When you are confident you can do it without a lot of fuss, repeat the process in your production site. Alert everyone that at 12:00 midnight on the 1/4 you are going to take your site offline. You can then place the site into Maintenance mode at that time, confident that no-one will be online. Do your upgrade and if you do not have non-standard blocks or filters or plugins, it should just upgrade easily. Make sure you have the right version of PHP and database server and everything... 

Average of ratings: Useful (2)
In reply to Colin Fraser

Re: restore 1.9 backups in 2.2 including user data

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi all

Colin said:

> The transition from Moodle 1.9 to v2.x cannot be accomplished easily when you have no opportuntity to make the kind of clean breaks discussed earlier. This then makes it essential to apply some planning to the process.

For example, may be you can close down the site during the festive season.

> Whatever is done, I suggest you set up a v2.2 and make sure you have all your courses in it before you do anything else.
>
> It can be done quickly, that is, select a date in the future, prepare a Moodle 2.x and parallel it with your existing v1.9.x, that is, make sure it has all the same courses and users all set up and on that date, everyone moves over on one day.
> ...

It may sound complicated, but you can have a "one click" upgrade easily under Unix. Well, it not not really a "click", executing a shell script I mean.
wink

#!/bin/sh
cd /var/www
rm -rf moodle2/*
/usr/bin/cvs -z3 -d:pserver:anonymous@SERVER.cvs.moodle.org:/cvsroot/moodle co -P -d moodle2 -r MOODLE_21_WEEKLY moodle > /dev/null
rm -rf /var/moodledata2/*
cp -ra /var/moodledata/* /var/moodledata2
rm /var/moodledata2/sessions/*
cd /root/bin
cp config.php /var/www/moodle2
/usr/bin/mysqldump --databases mdl_csd --user=SQLUSER --password=SECRET1 > mysqldump.sql
/usr/bin/patch mysqldump.sql mysqldump.diff
sed -e "s/http:\/\/my.domain\/moodle/http:\/\/my.domain\/moodle2/ig" -i.bak mysqldump.sql
/usr/bin/mysqladmin -pSECRET2 drop -f mdl2
/usr/bin/mysql -u SQLUSER -pSECRET1 < mysqldump.sql

The mysqldump.diff was created by editing mysqldump.sql manually to a copy and running

diff mysqldump.sql mysqldumpcopy.sql > mysqldump.diff

/root/bin/config.php is a manually edited version of moodle2/config.php.

This is not meant as a useful shell script: it is not parameteraized, no error schecks, etc. And I've annoymized it for this post too! But I hope you get the idea.

P.S. This is the example I've mentioned here: Upgrading from 1.9.12+ to 2.2 takes too long....
Average of ratings: Useful (1)
In reply to Visvanath Ratnaweera

Re: restore 1.9 backups in 2.2 including user data

by Colin Fraser -
Picture of Documentation writers Picture of Testers

But, Visvanath, not everyone uses Linux...

I still think best bet is to run it via an exact test install first, then again, then again, find out what can go wrong and how to fix it. This is the same as Tim suggests, but a more careful approach which is, I know, long winded and painful. Paralleling might be cleaner but is more intensive and the rapid one step change has its own problems. Essentially though, it is really up to the enterprise to make a decision that best suits them. For me, a careful and planned and consistent approach that is proven workable is a far better option.   

Average of ratings: -
In reply to Colin Fraser

Re: restore 1.9 backups in 2.2 including user data

by Genie Wu -

I did exactly what you described. I had the 2.2 installed, set a date to sunset the 1.9, and I am in the process of transferring everything over within a week.

One problem I came across is with the user password. I used the bulk upload to create users but the password will only work if I type in the encrypted password. The actual password doesn't work. Any suggestions?

I also plan to import the grades back to some of the courses. Would this work to re-establish student grades and enrollment?

Thanks in advance...

Average of ratings: -
In reply to Stephen Martin

Re: restore 1.9 backups in 2.2 including user data

by Susan Fleming -

Even in an academic environment there are occassions when student data from 1.9 backups need to be accessed.  We upgraded to 2.2.3 in July but before doing so we backed up and removed as many courses as we could to lighten the server load but now we need access to student data from some of the courses which were removed. 

Has anyone figured out how to restore backup files from 1.9 into 2.2 with student data intact?  If so, how did you do this?

Thank you in advance for any help,

Susan

Average of ratings: -
In reply to Colin Fraser

Re: restore 1.9 backups in 2.2 including user data

by Tiffany Morgan -

But it is possible to make a new course in 2.2 from a backup made in 1.9x?

So far I have issues getting this happen. Moving from 1.9.5 to 2.2 with a backup that excluded users and only included standard items (resources, forum, quiz).

I import and recieve an error that the selected file is not a standard moodle file. The restore process wll try to convert the file into the standard format and then restore it. When I continue it creates the course, but is unable to find any of the resources, quizzes or forums. Much like a user described here.

http://moodle.org/mod/forum/discuss.php?d=193192

Any ideas or help are very welcomed! smile

Average of ratings: -
In reply to Klaus Steitz

Re: restore 1.9 backups in 2.2 including user data

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

If you want to keep all your data, why not just upgrade your 1.9 site to Moodle 2.2? Then you will have all your data.

Or, if you just want some of your data transferred from 1.9 to 2.2, you can:

  1. Make a copy of your 1.9 site.
  2. Upgrade the copy to 2.2
  3. Use backup and restore to transfer courses with user data form your '1.9 upgraded to 2.2. copy' to you 'new 2.2 site'.
Average of ratings: Useful (1)
In reply to Tim Hunt

Re: restore 1.9 backups in 2.2 including user data

by Rosario Carcò -

T H I S

is the way to go safely, but you need a spare Server. That's exactly how I changed hardware and also Moodle Versions in the past. And I will repeat it to migrate our 1.9.12+ production server to 2.2x

The benefit of this procedure is, that YOU NEVER LOSE your running/production site. And you can test your new server for a couple of weeks before repeating the cloning and upgrading. And you can revert to your production server at any time. If you miss something you can still use your production server to recreate a backup-zip-file and recreate a clone to upgrade and so on.

OK to be exact: you need THREE servers at one time:

  1. production server (never touch a running system)
  2. clone where you install an exact copy of your production site and do an upgrade to the latest Moodle version
  3. new server where you do a clean fresh install of the latest Moodle Version. Then you copy the new backup-zip-files from the upgraded clone to it to restore them

If you go this way, you might save ONE or TWO hardware-servers:

  • install everything on the same hardware-server, if php-version, etc. meets the requirements of the latest Moodle version, and you have enough space on the data-disk
  • you need only to copy the two Moodle-directories containing the php-files and the data-files to separate directories, then import the database dump of the original server instance to your database changing the mdl_ prefix to something new and the moodle-database to another name too
  • then adjust the config.php file to reflect your new directories and add the new Moodle php directory to your web-server

And that's how it looks on my production server:

To do a clone and upgrade I will use a Virtual Machine on an ESX VM-cluster, just to produce the new 2.x backup-zip-files.

Rosario

Average of ratings: Useful (2)
In reply to Rosario Carcò

Re: restore 1.9 backups in 2.2 including user data

by Stephen Carter -

Hi Rosario


I am reading alot of forum posts at present as I attempt my upgrade form 1.9.X to 2.2.


I am doing something very similar to yourself using multiple servers, it is working out quite well.

However, there is an issue which you may not realise. 2.x does not produce backup-zip-files, it produces the new mbz file. To make matters a little harder, you cannot located the backup mbz files on your 2.x server. They are now hidden away for security purposes.

Whilst you can do a backup and restore, it is does take a while to manually transfer the mbz files betwee the two servers.

If I have missed something really obvious here please let me know.

Cheers

Stephen

Average of ratings: -
In reply to Stephen Carter

Re: restore 1.9 backups in 2.2 including user data

by Jan Eberhardt -

Like I said it's not the problem in the backup process. You're right, that the backups are now called "mbz" but it's nothing more than a renamed zip file... It also doesn't metter in the encryption, because that is done by the server himself. 

I located a zipping function in the class I mentioned before (in /backup/moodle2/backup_stepslib.php:backup_zip_contents.class). So I'll plan to encrypt the file right after zipping and then moodle can do whatever it want to that file. Hopefully moodle won't try to access the data saved in the mbz but that I'll check.

In conclusion my problem is, that I don't find the responding restore function...

Average of ratings: -
In reply to Rosario Carcò

Re: restore 1.9 backups in 2.2 including user data

by Rosario Carcò -

DEAR ALL... THEORY IST THEORY AND REMAINS NOTHING BUT THEORY until the day you proceed to ACTION...

The ICT-Directors board decided to outsource our Moodle Server to a Moodle Partner Firm and so all my plans to do it myself, following the steps mentioned above, were thrown out of the way. And I lost 30% of my job, which was my loved Linux-Apache-MySQL-PHP administration/development and know-how source I built up during the last 8 years.

I still could proceed to do it on my servers now out of production. But there are some problems to solve and that could not be solved:

  • there are lots of plugins/blocks/etc. you might have used on 1.9 not having any corresponding plugin/block/etc. on 2.x or not being capable of migrating/upgrading own 1.9 data. This would mean you had to deinstall these first. I guess you would have to produce clean backup-zip-files without that plugins/blocks/etc data, if you only wanted to import single important courses and not the whole bunch on your 1.9 server. In our case we returned to a standard 1.9 Moodle, being lucky that we did not install many third-party modules. We basically used my own myCourses and siteNavigation blocks, I still have to recode for 2.x, and nothing worth mentioning. One or two plugins did have 2.x versions but I think we deinstalled them on 1.9 to start over with the new versions, old data not being important for us to be migrated
  • WIKI is very critical and our Moodle-Partner knew how to copy most of it
  • Quizzes always were critical and our Moodle-Partner knew how to copy most of it. This does not mean that all of our 5'200 courses were copied/upgraded 100% reliably. There might still be problems no teacher discovered until now.
  • if your teachers copy/pasted from Word and other applications, e.g. from Mac-Computers, the course pages might be corrupted, as such HTML can not be displayed any more, even if it was no problem on 1.9 - In such cases you have to edit the html-text-fields using plain-text-web-forms or even editing the fields directly in the database, as I did many times when the GUI on the course-page was completely out of work.
  • Multi-Language-Tags seem to slow down the 2.x server more than they did in 1.9

I surly am forgetting a lot of other things, but the Moodle-Partner did a great job as they not only transferred data from one server to another as I had done many times when changing hardware, but they also transferred the database from mySQL to PostGreSQL and then upgraded the whole thing in little steps so as not to lose any bit.

After I saw this complexity, I was confirmed in my own plans to have tested everything on test-servers, trying to get backup-zip-files in the Moodle 2 mbz-format of the whole 1.9 clone, but I would have started on a fresh Moodle 2 server from scratch and would have imported only the really needed backup-zip-files after conversion to mbz-files. I would never have upgraded the whole server to 2.x even if I had succeeded many times playing and replaying the mentioned steps above. I would never have been able to guarantee a 100% success in such an upgrade and I would never have wanted to bear such a responsibility taking over old (and who knows, partially corrupt? data). BUT THIS IS WHAT THE MOODLE PARTNER DID in our case, so they are capable and willing to give this warranty, and I have to bend and kiss their feet smile

Rosario

Average of ratings: -
In reply to Rosario Carcò

Re: restore 1.9 backups in 2.2 including user data

by rosaria tufano -

Per favore è urgrntyissimo. C'è qualcuno di buona volontà che mi spieghi come effettuare un ripipristino da un corso modle 1 a moodle 2

grazie Rosaria

Average of ratings: -
In reply to rosaria tufano

Re: restore 1.9 backups in 2.2 including user data

by Rosario Carcò -

Cara Rosaria, come già spiegato prima, un file backup 1.9 non potrà essere ripristinato senza perdere buona parte dei dati degli utenti (user data). Se ho capito bene, soltanto il contenuto del corso può essere ripristinato. Secondo me la miglior soluzione resta quella di creare una copia esatta del sito 1.9 su un server a parte. Poi fare gli upgrades necessari per portarlo alla versione 2.x richiesta. Poi produrre i file backup mbz di Moodle 2 e finalmente ripristinare questi file sul server 2.x installato tutto nuovo, senza alcuni relitti.

Rosario (Saluti dalla Svizzera)

Average of ratings: -