upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Germán Alvarez -
Number of replies: 21

Hello,

There are currently some working Moodles with versions 1.x and 2.x.

I need to update to at least 3.11, ideally 4.0

Does anyone have a step-by-step for this that doesn't involve going through all the in-betweens?

Thanks.

Greetings.

Average of ratings: -
In reply to Germán Alvarez

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Sure. They are in the Moodle Docs:
- https://docs.moodle.org/19/en/Upgrading
- https://docs.moodle.org/20/en/Upgrading
- ...
- https://docs.moodle.org/311/en/Upgrading
- https://docs.moodle.org/400/en/Upgrading

You don't have to go through all the versions, but no hyper-jumps to the current stable (3.9, 3.11, 4.0) are possible from 1.x nor from 2.x. You can plan your route from this chart: http://www.syndrega.ch/blog/#php-and-dbms-compatibility-of-major-moodle-releases. See the column "Earliest from".

Read the footnotes, especially "Upgrade from 1.9 to 2.x broke so many things, it is advisable to rebuild the site - either through course backups and restore or create courses from scratch making use of the numerous new features added since then".
Average of ratings: Useful (4)
In reply to Visvanath Ratnaweera

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Germán Alvarez -
Thanks for the reply.

I think the best thing to do is install 3.11 or 4.0 and restore courses, the problem is that there are thousands of courses and users. Can this process be automated?

Thanks, again.

Greetings.
In reply to Germán Alvarez

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Ken Task -
Picture of Particularly helpful Moodlers

There are catch 22's (note that could be multiple) to that approach of restoring course backups from such old versions of moodle to a 3.11.highest or a 4.0.X+.

Plugins used in old sites ... were they critical ... is there a compat version of same plugin in the destination version?   Those plugins would have to be installed in the 3.11.highest or 4.0.X+ prior to attempts are restoring a course.

Type of course backups ... with users or without users.   Probably stand a better chance of success with *no user* backups restoring ... reason ... user roles conflicts ... let's remember you are trying *very* old to latest and greatest and there have been massive changes.

There is a command line utility knife that could restore via a looping shell script a bunch of courses to a category - category has to pre-exist so the new site would have to mimic the organization/categories of old site to end up with something similar.

So, theory ... old version of Moosh installed on old server, could backup courses to a folder on old server ... or you could get fancy and setup a mount point on old server that is on new server ... in position to restore courses then. 

Your 1.9 backups (having .mbz extensions) are zips - which makes the new server work a little harder cause it first has to convert those backups to version 2 backups before it attempts to restore.

Around version 2.middle of moodle - .mbz files became gzips .. tar.gz's - better compression and better restore.

And you will need plenty of time to work out a system, replicate that system so that your scripting works, then you could semi-automate.   If you had been through such a process before one might be able to do 4000 courses in 8-16 hours.   (Uhhh ... have done such a thing via moosh but not the hyperjump version backups and it did take that long!).

https://moodle.org/plugins/pluginversions.php?id=522

https://moosh-online.com/

It's a BIG job!   and can't help but think there will be some gotcha's that you will have to work through! :|

'SoS', Ken


Average of ratings: Useful (2)
In reply to Ken Task

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Ken Task -
Picture of Particularly helpful Moodlers

Follow up ... bottom line ... it's ok to WAFFLE ... change your mind again!

Are all these spread out over multiple servers?   And don't respond with a simple 'yes' ... be specific with info like OS (distro and version), PHP version + extensions, MySQL/MariaDB version and if DB server is on same server as code.

Plus free space.  Why is that a biggy?

At each hop upwards and after testing, one would  should make a minimal backup of code + DB.   Don't want to loose ground gained if the next hop doesn't go as planned one can do a site restore to the last known upgrade ... won't have to start over.

BIGGY ... the 1.9's to 2.x's - massive change to moodledata and how Moodle stores files ... so the early 1.9's to 2.x's need a FULL site backup IF ... IF ... you have to restore back down to  1.9 ... hopefully not!

Do answer these ...

How many 1.x's do you have?   What are their versions ... each one of them.

How many 2.x's do you have?  What are their versions ... each one of them.

Now my comment ... it's ok to waffle ... 1st suggestion was to march in place ... you looked at that an thought, no, course backup and restore to fresh would be the way to go ... but ... BIG BUT ... that strategy cannot work with all users and their data retained in restored course.  Re-read what I said ... *no user backups*.

Now waffling back to entire site *IS* ok!!!! ... like Emma suggest ... *only way* ... but she didn't mention how ... even a basic how ... well here's the basic 'how' - GIT.

IF ... IF ... I had been given this job ... no doubt I'd be using GIT!   Bash shell upgrade scripts with site backups just prior to the git upgrade - killing 2 birds - one stone!

For you, *side load* is first order of biz and perhaps a minor update to the current version of the 1.9's, the 2.x's, and the 3.x's *before* marching.

Once a site is successfully upgraded *and checked* ... plugins ... upgraded as well.

Site backup at that point.   Remember, we want a fall back on next hop just in case!

Another but ... versions of PHP + extensions + MySQL would/might have to be upgraded at the appropriate step in a march via git.

Ok, awaiting technical info/responses - true and accurate .. 'devil is in the details'!!!!

Or don't respond ... in which case ...?????   smile

'SoS', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Ri: Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Sergio Rabellino -
Picture of Particularly helpful Moodlers Picture of Plugin developers
My little contrib.

Definitely “it can be done” ad i’m running almost one moodle upgraded continously since Moodle 1.4.3.
But it waal done during the last 14 years, sometime was simpler, sometime was long in time. In moodle 2x i remember also epocal changes in assignments
After this work, after many try and fail steps, il you’ll get all of your moodle upgraded to 3.11.7 ( forgot 4x by now) get baci here and contribute to the community, i’m pretty sure you’’ll be a Moodle expert….
Lastly skilla needed: IMHO a good CS career with sysadmin and dbadmin knowledge, programminog languages concepts and direct knowledge of Web server/php/mysql/javascript develop and debugging, read many docs, moodle dev docs, so good english comprehension 
A “googler/forum asker “ won’t exits this job successfully.
Average of ratings: Useful (1)
In reply to Ken Task

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Germán Alvarez -
Thank you very much for the reply.

I will probably choose to install a 3.11 or a 4.0 from scratch and ask the tutors and teachers to reload their content in the new Moodle.

In this way, many old courses in disuse will be deleted and those that are still in use will renew their content if applicable.

At this moment there are two moodle 1.9, three 2.9 and the rest 3.x versions.

The problem of size is that, as in some educational institutions, the budget depends on the number of students and courses, so they never dismiss either teachers or students. Even if these have been received or have left the studies years ago. There are still courses of teachers who died.

For these reasons I think the best thing to do is to install 3.11 or 4.0 from scratch and "clean" the Moodles.

Databases vary between postgres and mysql. From now on it will surely continue with mysql.

Once again, thank you very much for the time you all took to respond and for your advice.
Average of ratings: Useful (1)
In reply to Germán Alvarez

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Ken Task -
Picture of Particularly helpful Moodlers

Mr. V has triggered some additional thoughts ... now that we know strategy to this point ...

Are you planning to consolidate all users/teachers/courses into one server?

You have:

2 instances of Moodle 1.9's - how many courses, how many students, how many teachers.

3 instances of Moodle 2.9's - how many courses, how many students, how many teachers.

we don't know how many 'the rest' means but ditto on how many questions above.

Depending upon those numbers, and if it is 1 server - that might have to be one beefy box! ... max'd out memory, a couple of TB attached devices for space.

Again, depending upon the numbers, you might be looking at really 3 servers ... 1 web interface (all courses/students/teachers), 1 dedicated DB server, and 1 dedicated server for /filedir/ of moodledata.

The specs of those servers pretty hefty ... especially memory/space.

OR ... OR ... OR ...

Am thinking those numbers are why you were mentioning docker.

So .... this to sugggest you have more planning to do! smile

'SoS', Ken

In reply to Ken Task

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Too many different things put together in one discussion:
- Marching a 1.9 site to a currently supported version is entirely different from marching a 2.9 site.

- Whether (some) sites will be merged or will the number of sites remain the same?

And completely missing: The additional plug-ins involved?

Well, if the teachers are to build the courses from scratch, then it is a fresh start. None of the above matter. Discussion closed! (I meant the original question. One can always discuss something, I know. Say, how to build courses will be a new discussion in the Teaching with Moodle forum. Or the performance topic in the Hardware and performance forum.) 
In reply to Ken Task

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Germán Alvarez -
I forgot to comment, all the Models are in different versions of Linux Ubuntu Server or Linux Debian.
From now on they will be on Linux Ubuntu Server and I am considering the option of using Docker and Kubernetes to improve performance. But that is a topic for another forum.
Thanks.
Greetings.
In reply to Germán Alvarez

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Ken Task -
Picture of Particularly helpful Moodlers

May as well continue here ...

Now that we know the entire situation, your strategy is a wise one.  Ubuntu LONG TERM SUPPORT server versions wise choice for moodles as you will find much of the docs in Moodle show Ubuntu.

Don't think I've ever seen performance as a reason to use a containerization (Docker).  Careful not to make your future admin of those servers more complicated than it needs to be. 

Since you've said MySQL ... MySQL Tuner is a good tool for *performance* of MySQL/MariaDB DB servers.

Since these are to be fresh installs, install using git a 3.11.highest.   Git will be by far the easiest way to install and the best for updating or upgrading later.

https://docs.moodle.org/400/en/Git_for_Administrators

Think I'd go no higher than a 3.11.x as 4 is still in 0 and a very new 'UX  experience' ... a 3.11.x will be familiar for teachers/students that have used moodle before ... less support for you to provide on things like "who moved my cheese'! smile  [IMHO!]

Another consideration ... teachers probably know they can backup a course and restore it.  Think I'd shut down the 1.9 completely to prevent them from attempting that.   The version 2 and 3 servers you have ... their backups/restores should work .... as long as you have the addons installed in the new.

If you are to leave the version 2 and 3 servers up and accessible, think I'd go into backup preferences at system level and make sure the backups teachers create would be no user, no events, no logs, no completion tracking, etc. course backups - basically course shells. 

''SoS', Ken

Average of ratings: Useful (1)
In reply to Germán Alvarez

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
So you've decided to ask the teacher to recreate the courses from scratch in a new Moodle? That is a generalization of the footnote "3. Upgrade from 1.9 to 2.x broke so many things, it is advisable to rebuild the site - either through course backups and restore or create courses from scratch making use of the numerous new features added since then" in http://www.syndrega.ch/blog/#php-and-dbms-compatibility-of-major-moodle-releases and is the most sensible thing for courses from Moodle 1.x and early 2.x. You have to handle the human factor though, the (still living) teachers might revolt.
wink

I too didn't know that Docker and Kubernetes improve performance. There is a dedicated forum for that topic called the Hardware and performance forum https://moodle.org/mod/forum/view.php?id=596. It is out knowledge base, but not much on Docker and Kubernetes there. Please do share your experience there.
In reply to Germán Alvarez

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
If you really have thousands of courses and users, I think I would disagree with the rest and recommend the upgrade path. You are going to have several steps but it is the only true way to keep all of your data and probably the safest. Hopefully you do not have many plugins and you will need to check those but, while it takes a little time, it is very doable. If your servers are virtual, I would recommend migrating at some point during the process to get latest os etc. While you can restore courses from older versions, I do not believe that you can restore courses with the data and users from 1.x versions.
Average of ratings: Useful (2)
In reply to Emma Richardson

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Yes, if "really have thousands of courses and users" the official https://docs.moodle.org/en/Upgrade path is the best, provided that there are no complications with plug-ins. This can be solved by dropping/replacing the plug-ins during the upgrade march. But how does one merge many Moodle instances in to one? That is probably what the OP is asking. If the Moodle instances are remain separate, then solvable, but so many instances times a march over so many major releases is going to be quite some work as others have said.

And yes, the target should be a supported 3.x version, means either 3.9 (LTS) or 3.11. Generation 4.x is "future".
wink
Average of ratings: Useful (1)
In reply to Visvanath Ratnaweera

Ri: Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Sergio Rabellino -
Picture of Particularly helpful Moodlers Picture of Plugin developers

Before thinking on consolidation, the first goal is to have all the instances at the same level. Then you could setup/mimic the moodle network (term a little abused) link between instances that can be browsed seemlessy mantaining each single instance in a virtualhost. 

Having moodle uptodate could not be  hard requirement, to say the truth, but almost an LTS is needed to work into the wild.

This is only one of  the possible strategies.

Average of ratings: Useful (1)
In reply to Sergio Rabellino

Re: Ri: Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Germán Alvarez -
thank you very much for the time you all took to respond and for your advice.
In reply to Visvanath Ratnaweera

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Germán Alvarez -
thank you very much for the time you all took to respond and for your advice.
In reply to Germán Alvarez

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Saw that you've asked the same question in the Spanish forum: https://moodle.org/mod/forum/discuss.php?d=435442#p1752198. Please put a note there concluding that you have got the answer in this discussion.
Average of ratings: Useful (1)
In reply to Visvanath Ratnaweera

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Germán Alvarez -
Thanks for reminding me of that! I'm going to write it. Greetings.
In reply to Emma Richardson

Re: upgrade Moodle 1.x, 2.x to Moodle 3.11 or 4.0

by Germán Alvarez -
thank you very much for the time you all took to respond and for your advice.