Moodle migration

Moodle migration

James Steerpike -
回帖数：28
Particularly helpful Moodlers的头像

A recent post has highlighted at least to me the 1500 words in our migration docs may not be straightforward to everyoneattempting to move to a new server. A review may be a good idea.

I do have a couple of questions.

The recommended process involves database dumps, database restores, copying of directories between servers and changing config flies. Is the process of backing up your courses and restoring them to a new server a viable option for a small scale Moodle as both these actions can be carried out within Moodle itself? Would plugins be a problem? If such a transfer is possible,  is it possible to move courses from an earlier Moodle version to a later version?

平均分： -
回复James Steerpike

Re: Moodle migration

Ken Task -
Particularly helpful Moodlers的头像

As with all things moodle - depends ....

Define "a small scale Moodle"!

One of the reasons for a migration is lack of space with current hosting.  Default for course backups is to store in sea of files moodledata/filedir/ - not a designated directory where one could find them easily and thus be able to transfer them to new server.

Happen to be attempting to assist an Uni right now that has no space left on current server and that has been hacked.

Age of backup?  Backups from older version of Moodle to newer is preferred.   If last backup was made a month ago, restore of that course to new server, users might have to re-do what they have already completed.

New server has all addons contained in courses installed and configured?   3rd party themes do fail gracefully (most of the time).

Full course backups with users or no users?

New server: same authentications/enrollments?  Duration of allowed enrollments.   SMTP setup?   Notifications tested?

Size of course backups.

Newer versions of moodle restore in background is now default ... get into too much of a hurry and could end up with many 'pending' course restores.

Depending upon number of courses, moosh could be your friend ... or using the cli course restores in a looping bash shell script.   Course backups would have to have been uploaded to a file system repo.

'SoS', Ken

平均分： -
回复Ken Task

Re: Moodle migration

James Steerpike -
Particularly helpful Moodlers的头像
Moosh certainly is interesting. With lack of space, it would be possible to use it to loop through courses, backing them up, using scp to send them to the new server, and then deleting the backup from the old server.
平均分： -
回复James Steerpike

Re: Moodle migration

Ken Task -
Particularly helpful Moodlers的头像

The killer here is 'lack of space'.   Am currently working with a Uni that used Bitnami (ugh!) hosted on Amazon in that situation.

The other killer is 'shared hosting' - inability to do much at all.

A migration consist of 3 parts: backup, transfer, restore.   Sounds simple - site or courses.

Site and course backups and transfer could be accomplished in one step IF ... IF ... where the site is currently hosted OP had command line and could mount a directory on destination server.

Both the backup script that comes with Moodle in admin/cli/ and moosh as well, have a destination parameter.   That destination parameter could be the mounted directory on the destination server.

Next best is rsync.

Least desired, but on shared hosting is the only option, download then turn around and upload to new server. 伤心

IF moving from shared hosting to VPS on same network/with same provider, the provider might be able to set that up.   Doesn't hurt to ask ... even though the likelyhood of that is almost nil. :|

Setting up a mount point is way to advanced for many Moodle Admin OP's and is somewhat specific to distro.

'SoS', Ken

平均分： -
回复Ken Task

Re: Moodle migration

Ken Task -
Particularly helpful Moodlers的头像

Replying to self just to keep together ... how sick is that! :|

Described above is what I would call (in general) a 'push' - old server to new server.   There is yet another way ... that I generally call a 'pull' and might require repeating.

On old server create a web accessible directory called 'transfer'.  It's to that directory site backups or course backups are stored.

On new server, one uses a command line utility called 'wget' or similar - a text based browser would also work (lynx/elinks) ...

On new server a 'transfer' directory and in there a 'getlist' which is a text file with one liners - url to the backups in shared hosting /transfer/.

Example: in shared hosting, an archive of moodledata/filedir/ is made and saved to /transfer/ as moodledatafd.tar.

The url would be https://oldhost/transfer/moodledatafd.tar

In the getlist: one line would be that URL.

Ditto for sql dump or archive of code directory.

On new server:

wget -i getlist

wget would read each line in the getlist file and access it ... tranfering that file to new host.

Goes without saying one should remove files in old host transfer as soon as possible.

'SoS', Ken

平均分： -
回复James Steerpike

Re: Moodle migration

Visvanath Ratnaweera -
Particularly helpful Moodlers的头像 Translators的头像
Hi James

They are two different things, service different purposes.
 
- Moodle migration
This is in fact Site backup followed by the Site restore of the same. It give a copy, we often call a clone, of a whole site in one transaction of a bag of three items.
 
- Course backup and Course restore
This copies a single course. And you transfer only one item. Of course by repeating it for every course, you can copy a whole site. I don't want to do that, unless there's an emergency like the database is gone but can find a collection of course backups somewhere. This is more often used to recover a single course. Say a "teacher error" destroys a course. And you have a backup taken before the incident. For that I don't want to restore a whole Site backup!
 
So both need to exist.
平均分： -
回复Visvanath Ratnaweera

Re: Moodle migration

James Steerpike -
Particularly helpful Moodlers的头像

Certainly both need to exist. However https://docs.moodle.org/405/en/Moodle_migration refers to only one method for when  "you need to move your Moodle site from one server to another. " Backup and restore can be done by web browser and a down and up load so looks technically easier for a site with a handful of courses. It does not scale for larger sites but should be mentioned as a  link from migration to backup and restopre

平均分： -
回复James Steerpike

Re: Moodle migration

Visvanath Ratnaweera -
Particularly helpful Moodlers的头像 Translators的头像
Sure - the Moodle Docs are open for everybody with an account on moodle.org to edit. As you yourself found out, every wiki page has a "talk page" like Talk:Moodle_migration. We can't be discussing every, thousands(?), of pages - each in 35 versions - in the forums!
平均分： -
回复Visvanath Ratnaweera

Re: Moodle migration

James Steerpike -
Particularly helpful Moodlers的头像

I will stick to talk then and see how many people actually watch pages...

平均分： -
回复James Steerpike

Re: Moodle Docs: Watchers and Talk pages

Visvanath Ratnaweera -
Particularly helpful Moodlers的头像 Translators的头像

Now, you too are finding out that the whole concept of Moodle Docs is flawed. The belief that random moods of a random population will compile a continuously spawning documentation for a broad and ever growing piece of software like Moodle is tantamount to the Infinite monkey theorem. 

I've written (too) much there and not available for any further discussions on that topic. If you are interested, you'll find them in the Moodle community sites forum which absorbed the Moodle Docs forum.

BTW, Australian Mathematicians Debunk 'Infinite Monkey Theorem'

平均分： -
回复Visvanath Ratnaweera

Re: Moodle migration

James Steerpike -
Particularly helpful Moodlers的头像

Late at night here but your comment ( and the rather puzzling Vanessa post) got me thinking

First, copying and pasting of bash commands you don't really understand depends on trusting what someone else has given you. In particular, running under sudo can trash every application and even LInux itself in just 8 characters.

Secondly, anyone can log on and change any community page.

Not so smart after all...

平均分： -
回复James Steerpike

Re: Moodle migration

Visvanath Ratnaweera -
Particularly helpful Moodlers的头像 Translators的头像
> First, copying and pasting of bash commands you don't really understand depends on trusting what someone else has given you.

"Trust" I don't know. But again, there are many aspects.

> In particular, running under sudo can trash every application and even LInux itself in just 8 characters.

Yes, that is definitely one aspect. The command, especially run with superuser privileges, may cause damage.

Then there is the "Vanessa effect". The command writes "vanessa", where "jammy" is expected. That didn't damage the anything, but didn't give the expected result either. And for the audience these copy-and-paste commands are intended to, they wouldn't see top nor bottom! So back to Ken. 眨眼
平均分： -
回复Visvanath Ratnaweera

Re: Moodle migration

Ken Task -
Particularly helpful Moodlers的头像

'back to Ken'!!!???

"anyone can log on and change any community page."

I've already thought about doing that with what I saw suggested about 'trying' ... 'just to see if...'.  Harder to reverse engineer!  Who wants to go one step forward just to go 2 steps back?

Thought moodle was about teaching and learning - not trial and error - but, then again, that's how some folks learn - but only the ones that have a high failure quotient.

Then there is 'trust' ... maybe a F-2-F online Zoom or Google Meet?

There is also, **hire** someone to do ....

So again ... I have no answers that are 'one size fits all'!

'SoS', Ken

平均分： -
回复Ken Task

Re: Moodle migration

Visvanath Ratnaweera -
Particularly helpful Moodlers的头像 Translators的头像
This is insightful.

> I've already thought about doing that with what I saw suggested about 'trying' ... 'just to see if...'. Harder to reverse engineer! Who wants to go one step forward just to go 2 steps back?

Wisely put.

> Thought moodle was about teaching and learning - not trial and error - but, then again, that's how some folks learn - but only the ones that have a high failure quotient.

Ha, ha, ha! A great qualification to be a pedagogic - in my eyes at least. And politically correct! 
眨眼
平均分： -
回复Ken Task

Re: Moodle migration

AL Rachels -
Core developers的头像 Particularly helpful Moodlers的头像 Plugin developers的头像 Testers的头像

> Then there is 'trust' ... maybe a F-2-F online Zoom or Google Meet?

On numerous occasions I have used various forms of F-2-F online using Skype, Zoom, Messenger, Team Viewer, etc., and greatly prefer any one of these when the language is not a problem. My preferred go-to is Skype mostly as I maintain the Skype Moodle plugin. I would really like to see about trying out the new instant language translation available on some of the latest PC's and phone's.

Due to also maintaining Hot Question, MooTyper, Diary and now iContent, I have also carried on a lot of email conversations via Google Translate.

平均分：Useful (1)
回复James Steerpike

Re: Moodle migration

Rick Jerz -
Particularly helpful Moodlers的头像 Testers的头像
I just went on to "Moodle Migration" and hopefully made a few more "improvements." I keep trying to integrate the ideas that everyone has noted. I will give it a few days to settle down and then revisit it again in the near future.
 
There is a "Quick and hacky method" that I didn't touch.  However, I wondered if there should be a "The GIT method" section.
平均分：Useful (1)
回复Rick Jerz

Re: Moodle migration

Ken Task -
Particularly helpful Moodlers的头像

Edit only that which you know ... don't remove something you don't!

"However, I wondered if there should be a "The GIT method" section."

Nope!  There isn't a 'git method' in migrating.

'SoS', Ken

平均分： -
回复Ken Task

Re: Moodle migration

James Steerpike -
Particularly helpful Moodlers的头像
The git is usually a hidden folder so a copy from one git controlled Moodle folder to another maintains git history. But I think the migration page is long enough without the complications. of maintaining updates by git.
平均分： -
回复Rick Jerz

Re: Moodle migration

Ken Task -
Particularly helpful Moodlers的头像

Question: is the assumption that the destination server is a VPS and has a cPanel and phpmyadmin (which is a common combo)?  Or a Plesk with phpmyadmin?

If one follows directions on importing SQL dump file into a new DB (as well as creating the new DB with proper character set/collation) will phpmyadmin see the DB?

If it doesn't, why didn't it?

平均分： -
回复Ken Task

Re: Moodle migration

Visvanath Ratnaweera -
Particularly helpful Moodlers的头像 Translators的头像

Wonderful to see the hard-core team of instruction authors. If I'm an authority, I would channel your energy in non-conflicting paths. A bunch of star ballerinas together on the stage won't give a great show.

平均分： -
回复Ken Task

Re: Moodle migration

James Steerpike -
Particularly helpful Moodlers的头像
It does seem to assume a VPS and may deter some users who can migrate using GUI database and file operations.
平均分： -
回复James Steerpike

Re: Moodle migration

Visvanath Ratnaweera -
Particularly helpful Moodlers的头像 Translators的头像
> may deter some users who can migrate using GUI database and file operations

What about the other end of the spectrum? See:
This article is written with the assumption that Moodle will be hosted on a single server. But what about large Moodle installations that are hosted on multiple "load balanced" servers? These sites may share the Moodledata folder using e.g. NAS storage. The database may also be hosted on a different database server.
Talk:Moodle_migration 17:51, 25 April 2016 (AWST)
平均分： -
回复Visvanath Ratnaweera

Re: Moodle migration

James Steerpike -
Particularly helpful Moodlers的头像
Migration isn't that difficult. Move two directories and a database, then change a few config lines. Users with little technical knowledge will need guidance but a system built for performance and large numbers of users should have specialist IT staff. I admit I have a specific user in mind - a teacher with no formal IT training on a limited budget but willing to learn and with ambition limited to a few tens to a few hundred students on Moodle.
平均分： -
回复James Steerpike

Re: Moodle migration

Visvanath Ratnaweera -
Particularly helpful Moodlers的头像 Translators的头像
> I have a specific user in mind - a teacher with no formal IT training on a limited budget

Then that must come before the answer. Otherwise the reader doesn't know. All sorts of people consult Moodle Docs.

I know, you did that in the case of the Rewrite of Ubuntu Step by step. But this, Moodle migration, is a different discussion.
 
> willing to learn and with ambition 
 
I doubt that. Copying and pasting Unix commands is not learning anything - except maybe the hard way (through small explosions and vapourized data). We spoke about some cases in the Ubuntu step-by-step discussion.
平均分： -
回复Visvanath Ratnaweera

Re: Moodle migration

James Steerpike -
Particularly helpful Moodlers的头像
Does it matter if I changed the Step by Step?
From asking the user to edit with vim, the world's most unfriendly text editor and then getting the instruction to switch into Vi editor mode wrong, leaving the user to wonder why key presses don't work.
Unnecessary steps such as setting the default search engine, not necessary since mariaDB was at 10.3
And then a failed install because php 7.4 was loaded (through an added repo) and Moodle 4.5 requires 8.1, the git points to the wrong Moodle and in the 3000 words of the install it doesn't bother to tell the userf to change the max vars. What specific user would find that a useful document?
It does require changes but I am not seeing specific suggestions in the talk page. The article has an edit and a revert button for any logged in user.
Migration I did change. it seemed to me someone asked to migrate would not always know a new "server" is a web server, database and php all at the correct versions and extensions.
I can only edit what I know. I don't have a shared hosting account, a Mac, or a NAS.
平均分：Useful (1)
回复Visvanath Ratnaweera

Re: Moodle migration

Emma Richardson -
Documentation writers的头像 Particularly helpful Moodlers的头像 Plugin developers的头像
I disagree wholeheartedly with this as this was how I started with linux. Most people want to know what commands are doing and will at least google the command or find other posts that give the same commands and then try to figure out why...I now know enough about linux to know what commands do but I still today do a lot of copying and pasting...the reason I kept the debian page updated was because I used to use it every time I built a new moodle server...I haven't built a new one in a while which is why it is outdated!
平均分： -
回复Emma Richardson

Re: Moodle migration

Visvanath Ratnaweera -
Particularly helpful Moodlers的头像 Translators的头像
Sure!

===
Q. Copying-and-pasting Unix shell commands on the terminal is good.

[ ] True [ ] False

A. Of course True. It was good *for me* and is still good *for me*.
===

Yep, that is the (digital) world of Moodle (Quiz). The analog world is unfortunately a bit more complicated. The answer there is True and False. Why? Because there are people in the audience who tick that way AND who don't tick that way. So the question there is: Are you writing those instructions for yourself or for the others? Or are you assuming that the world is good for you is good for everybody?
平均分： -
回复James Steerpike

Re: Moodle migration

Ken Task -
Particularly helpful Moodlers的头像

First, I'll answer my own question with a little story ...

In an OP's server (don't recall provider) and OP desired to clone a production site to have a development site.  Cpanel did provide Terminal and that's where I did the cloning.

Created the new DB via command line.   After getting the clone up and running checked phpmyadmin because that was a tool OP would still use.   In phpmyadmin, the DB I created from the command line didn't show!!! ????

Why?   When I created the DB from command line I didn't include the customer prefix in DB name.   Footnote to a migration?

Reasons for a migration:
On shared hosting and site has grown beyond the limitations of shared hosting.
Most common?  Memory, Space, Inode limits, other?

Then one gets off into how much ... memory, space, inode limits.  There does not exist a calculator app.  More means more $ spent. And I find myself giving strange advice ... 'better to have more than enough than not enough'!  What the heck does that mean!!!! :|

Devils are in the details!

Moodle Docs show mysql commands.

At one point in time those commands applied to both MySQL and MariaDB.
Defaults for character set/collation - now industry standard utf8mb4, collation has to match character set - but doesn't have to be utf8mb4_unicode_ci - config.php file comes into play.

MySQL
https://dev.mysql.com/blog-archive/new-defaults-in-mysql-8-0/

MariaDB
https://cpanel.net/blog/announcements/mariadb-server-to-become-default-database-in-cpanel/

From moodle docs:
mysqldump -h example.com -u myusername -p'mypassword' -C -Q -e --create-options mydatabasename > moodle-database.sql

Another option would be to use a tool like phpMyAdmin to manually make a backup. 

-h may not be needed ... localhost.
The switches: -C -Q -e --create-options?

In mysql 8 are SSL/TLS now - but that's not really needed if DB is localhost.

Example of a command used in both MySQL and MariaDB
mysqldump

mysqldump is used to dump a database or a collection of databases for backup or transfer to another database server.

From MariaDB 10.5, the client is called mariadb-dump. It can still be accessed under its original mysqldump name via a symlink in Linux, or an alternate binary in Windows.
MariaDB starting with 11.0.1

From MariaDB 11.0.1, mysqldump (the symlink) is deprecated and removed from the mariadb"

Reference: https://mariadb.com/kb/en/mysqldump/

When OP decides to migrate to a VPS, the new server is spun up by provider. The account from shared hosting is copied to new VPS.
Translation: the customers environment is still in a user jail:
/home/accountid/ - even though /var/www/ exist.
That copy handles moodle code and moodledata but not the DB itself.

There is also the issue of old moodle code -> new VPS and the versions of PHP + extensions available on new VPS too high to run the older moodle code.

The bottom line ... harder to provide accurate docs on a Migration.  Footnotes per provider if provider diffs are known.  Related .. Mr. V's chart ... has footnotes ... nice to have but how many folks click on the footnote link?

So somewhere a disclaimer needs to be shared with readers ... may not work with all providers.   And then to be complete, maybe separate providers pages that cover the diffs! :|. Wow!  Mega!

'SoS', Ken

平均分：Useful (1)