Moodle in English: Moodle Install and Migration

Hello everyone,

I hope you can help me. Little background: I have my own CentOS Linux release 7.9.2009 (Core) server and have been building Wordpress sites for some years, but have no experience with Moodle.

One of my clients / turned permanent employer wants me to host the moodle install (currently with a 3rd party) as well as the website I build and host for her already. I made some initial enquiries, and saw that moodle runs on mysql and php seemed like not such a stretch to figure it out to be able to migrate the whole thing over and my server should be able to handle it easy.

I believe I have all the correct specs on the server, I have checked (versions, extensions etc.) - so tech requirements are met.. i am almost sure

I have a fresh hosting account and a clean database. I have CLI access but zero experience there

The third party currently hosting the Moodle install will not give me any information whatsoever, to help me make the transition - only what is included and not included in the backup - see screenshot.

I have admin login to the front end of the Moodle install in question and can see the version they are running is 3.9.2+

I asked if they could upgrade to latest version before export, they said no.

So I need help with Installing this 3.9.2+ version on my server, I have WHM/cPanel/Terminal access

and also understanding what it means for us if those not included files in the screenshot are dropped.

Will the student experience change? Is this just standard procedure/language?

Is there a way for me to see what exactly is going to be dropped and install myself?

Thank you for any advice,

Lauren

Attachment screencapture-moodle-sussexbusinessschool-admin-environment-php-2021-07-09-08_45_26.png

Attachment Screenshot 2021-07-17 at 08.19.33.png

Average of ratings: -

Re: Moodle Install and Migration

by Visvanath Ratnaweera - Saturday, 17 July 2021, 5:57 PM

To migrate a Moodle instance don't create a new one. Do a https://docs.moodle.org/39/en/Site_restore instead.

Since you don't get the Moodle source code from the former party, get it as described in https://docs.moodle.org/39/en/Git_for_Administrators#Obtaining_the_code_from_Git.

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Monday, 19 July 2021, 3:42 PM

Thank you Visvanath. Just settling in to read all the replies and hopefully get this moving.

Average of ratings: -

Re: Moodle Install and Migration

by Leon Stringer - Saturday, 17 July 2021, 6:49 PM

Version

Moodle 3.9.x is the current long-term support (LTS) release supported until May 2023 so it makes sense to stick with that for now at least. But 3.9.2 is 10 months behind the current release, 3.9.8. I would use the 3.9.8 source code on the new server and apply this update as part of the migration process.

Migration

Along with the link Visvanath provided, I would also point you to Moodle migration although obviously you won't be copying the source code. A Moodle site consists of: 1) the source code, 2) the database, and 3) the data files ("Moodledata") which contains any files for the install, for example student assignments, corporate logos and additional languages. To migrate the site these three items must be copied.

The source code should be fairly standard (see next section) so can be obtained from Moodle.org except for the site-specific configuration details in config.php. Since this file won't be available you could create the settings manually based on config-dist.php. Instead it would be easier to install an empty Moodle site on the new server so config.php gets created, then change the settings for Moodledata ($CFG->dataroot) and the database ($CFG->dbname, etc.) to point to the migrated items.

Will the current host provide the packaged data twice? With the first copy you can do a test migration allowing you to 1) test the process and resolve any issues, and 2) hand it over to the customer for them to check prior to migrating the production site.

Plugins, themes and core code changes

Moodle supports plugins which are generally either:

Core plugins: included as part of the Moodle source code downloaded from Moodle.org
Additional plugins that are publicly available: the Plugins directory is the main repository for these but they may be available from different sources, for example: provider's web sites or GitHub.
Additional plugins that are not publicly available: these may be available either on a paid-for basis or only available when hosting with a specific provider.

Check what additional plugins are installed on the site (Site administration ▸ Plugins ▸ Plugins overview, click Additional plugins). For each additional plugin see if it's in the plugins directory or search the web to see if it's available. If not then your customer is likely to lose that plugin's functionality after migration.

Themes are a type of plugin. It looks like the site is using Remui which is not publicly available so the appearance of the site is going to change. You may want to look into alternative themes or advise your customer to.

The host's list of what's not included suggests there could be core code changes customising Moodle's functionality. Since they're not making these available we cannot say for certain if there are or what they do. But if there is any customised functionality that your customer is using then they're going to lose that on migration.

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Monday, 19 July 2021, 6:18 PM

Thank you Leon.
I have a question about the dataroot permissions. I think I may have done it wrong.. I have been reading all the docs and created a directory "moodledata" one level up from the web root like this:
# mkdir /path/to/moodledata
# chmod 0777 /path/to/moodledata

but I think I may have done it from the Terminal in WHM, ie as a server "root" rather than the specific hosting account user "sbs2108x2"

do I need to change this? maybe I am overthinking it now, but wondering if I talk the current supplier into migrating from server to server by giving them ssh access to the account, they won't be able to write to that folder.... I dont particularly want to give them root access to the whole server as have other customers on there

they may refuse to do this anyway, but I am going to try asking (have a call with them in a couple of hours)

regardless of whether they agree or not, do I need to change that permission from "root" to "sbs2108x2" for moodle to work?

Thank you for any thoughts on this!
Lauren

Attachment Screenshot 2021-07-19 at 11.11.18.png

Average of ratings: -

Re: Moodle Install and Migration

by Leon Stringer - Monday, 19 July 2021, 6:31 PM

I think you're correct, moodledata should be owned by the hosting user.

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Monday, 19 July 2021, 7:06 PM

thought so.

is there a quick way to update this with CLI? if I log back in at root level terminal window assuming I can only change from there now

I will try and google this, but so much information, in case it a simple comand that you know pls share

thank you!

Average of ratings: -

Re: Moodle Install and Migration

by Leon Stringer - Monday, 19 July 2021, 7:23 PM

The command would probably be chown -R sbs2108x2. /home/sbs2108x2/moodledata (the '.' after sbs2108x2 is intentional).

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Monday, 19 July 2021, 8:26 PM

Magic! thank you <3

Average of ratings: -

Re: Moodle Install and Migration

by Rick Jerz - Saturday, 17 July 2021, 10:17 PM

I am a little off target, I am not an expert on this topic (refer to Visvanath, Leon, and others,) but for background, maybe some of my videos will help you as you are getting oriented and educated.

See https://moodle.org/mod/forum/discuss.php?d=401983#p1621924

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Monday, 19 July 2021, 6:21 PM

thank you Rick, I will have a look. It is all rather overwhelming... looking like my WordPress front end experience isn't really enough to grasp the entirety of this migration... Hanging in there. learning

Thanks again.

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Saturday, 17 July 2021, 10:26 PM

Am going to add 2 cents because I have helped persons who were hosted with a Moodle Partner, migrate to a local/inhouse server (stock) moodle before.

It's an adventure and might involve some direct manipulation of DB tables and rows. Hopefully the admin interface will handle it but ...

Install a git acquired 3.9 in another domain.
Get that running ... add no plugins ... it's just a stock 3.9.highest

Mine is a 3.9.8 (Build: 20210712)

in code/admin/cli/
a bash shell script called 'getables'

mysql -u root -p'[password]' -e "use moodle39;show tables;" > 39tables.txt; cat 39tables.txt|more; wc -l 39tables.txt

chmod u+x gettables

execute: ./gettables

Will generate a txt file of all the tables present in the db (moodle39 in example) and a count of lines in the txt file at the end ... which is off by one (first line in the text file is a comment: Tables_in_moodle39)

468 39tables.txt

Do the same thing with the DB dump they gave you ... only name the txt file differently ... 39-cust-tables.txt

You can then compare the two with a diff command.

Now the admin interface of the code you use with their sql dump should show missing from disk ... true, you don't have the code directories/plugins they installed. And if you upgrade the DB those tables should be removed, but ... maybe not. Above might help get your DB for new site cleaner/meaner.

You might have to drop tables ... or directly manipulate rows in tables. Theme comes to mind.

Tables that contain config
mysql> show tables like '%config%';

of those, the following you might have to manipulate rows ... removing rows.

mdl_config_plugins

Moodle logs everything ... your largest table will probably be that 'who done it' table. mdl_logstore_standard_log

Ask customer how often they used the search for activity.
Since there will be rows that reference things no longer present in the new site, why keep them?

Truncating that table will reduce the overall size of the DB a lot ... and it will start over. It will make site faster!

Again ... 2 cents more.

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Monday, 19 July 2021, 6:32 PM

Hi Ken, Thanks do much for all the advice. I did actually start doing this like you are suggesting as an exercise to see whether I can install a fresh copy of moodle before attempting the migration. I didn't get very far.. I uploaded the latest moodle version via ftp and tried to run the install with CLI (as using the same domain as it is currently in without DNS update) but got warnings errors and didnt even know what to do next or how to exit terminal or anything. I think this is all a bit too much for my skill level

*sweaty face emoji!

I ended up wiping the hosting account and starting again as couldn't delete everything manually....
Now I am going to beg the moodle partner to migrate to our server with ssh... have a call with them in a couple of hours..
What are your thoughts on the "moodledata" folder permissions - did I fluff that up too? I asked in a reply to Leon above..

Anyway, its just lovely to have you all attempting to help me! Such a contrast to what the current partner is doing LOL One way or another I am sure I will get there.
Have a great day!
Lauren

Average of ratings: -

Re: Moodle Install and Migration

by Rick Jerz - Monday, 19 July 2021, 8:23 PM

Moodle can be installed in many ways. I showed one way in my video that I referenced.

As you are learning Moodle and how to install it, you might want to try installing a local version on your computer. The more experience that you gain, the less sweat will occur. 😓 However, some of this might be beyond your skill level. It's up to you to judge this yourself.

MoodleMoot2019 (Global and Mountain), "How to Install a “Sandbox” Moodle on Your Windows PC/Mac"

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Monday, 19 July 2021, 8:59 PM

Hi Rick, Thank you, I completely agree, best way to learn anything is by doing it. I am a little stressed but only because the customer wants it moved by 1st of August

which is when the renewal is with 3rd party, so I am kinda trying to learn in 3 days what people learn in years! Challenge accepted! haha

Average of ratings: -

Re: Moodle Install and Migration

by Rick Jerz - Monday, 19 July 2021, 9:51 PM

At least you understand that a good place to start is with a fresh install. If you cannot have success with a fresh install, don't consider a migration. One thing (among others) that complicates a migration are extra plugins that the original site might have, so be careful.)

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Monday, 19 July 2021, 10:08 PM

I think it would help if you could share some more info about where this site is to be hosted (what provider) and some other info which isn't hard to collect.

First provider ... who? See your address is .uk so is provider in UK? OVH, other?

Why? Many providers do have FAQ/Customer Community sites just like this one only specific to hosting provider.

You have terminal ... how did you get there? An icon in a panel? (cPanel). Not all panel's are a like ... some more restrictive than others. As far as terminal, appears when you use that you are in a 'user jail' ... and may/may not be able to 'su' (substitute user) to root user.

So, get into terminal and issue the following commands:

whoami

pwd - present working directory ... will show a path.

ls - list files/directories (the term directories = folders)

uname -an - will show system information ... for example on one of my servers:

Linux server 3.10.0-1160.31.1.el7.x86_64 #1 SMP Thu Jun 10 13:32:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Shows the system is using el7 - which is CentOS Enterprise version 7 - which isn't ubuntu or debian and while most of the basic commands (which you may/may not have access to) are the same, there are some for installing software that would be different.

which php - shows the path to php-cli

And using that full path:

/usr/bin/php -v

-v is version for php-cli .... example, mine shows:

/usr/bin/php -v
PHP 7.3.29 (cli) (built: Jun 29 2021 09:30:31) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.29, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.3.29, Copyright (c) 1999-2018, by Zend Technologies

whereis git

shows if 'git' is installed on system and it's path:

whereis git
git: /usr/bin/git /usr/share/man/man1/git.1.gz

Note, above does show there is an online manual for git.

man git - shows the online manual - for git ... more info than one would want to learn at this point, but nice to know it's there when/if one needs to learn more about git (or other commands).

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Tuesday, 20 July 2021, 9:10 PM

Thanks Ken,

and everybody!

It is really nice to know I have some support from you guys if I run into problems! Gives me the confidence to just do it!

I will be given a DT back up and moodledata backup next monday, and have to make it happen on the day, and update DNS the following day (moving to the same domain on diff server) I have access to domain control and totally cool with DNS etc side of things.

I have my own DS with Web Host Manager (WHM) and cPanel. I have full access to everything, i can give accounts shell or jail shell or no shell etc, and also I have support as well if I need it. I asked them (UKFAST) to install all the php extensions that moodle needs that I couldn't find for example.

I am pretty confident that my environment is more than suitable and I have full control of my dedicated server. I am only lacking hands-on experience with moodle

but looks like I will get that next Monday. The client wants maximum 48 hours downtime. Fun times here

So after all the reading so far I have a plan of action:

Spend the weekend installing a sandbox on my Mac and get it working as a practice exercise!

Come Monday 26th:
1. Download Moodle 3.9.2
2. Place into public_html
3. Restore moodledata backup into an appropriate directory
4. import DT with phpMyAdmin
5. update config.php with new DT credentials and point to moodle data dir
6. update host files so I can see the install (as moodle.sussexbusinessschool.com will continue to point to prev provider until I change DNS after I am happy it works!)
7. login /admin and install the theme (have .zip) - hopefully it will let me do that even though I am getting in there with host files updated pointing to my server... haven't tried that before...
8. Install any other missing plugins under / additional plugins
9. check all is looking/working as it should
10. update DNS

Phew.

Anyone see any holes in my plan? Any thoughts very welcome.

Thanks all,
Lauren

Average of ratings: -

Re: Moodle Install and Migration

by Rick Jerz - Wednesday, 21 July 2021, 12:32 AM

Yes, I might be seeing some "holes."

1) Why wait until Monday 26th? You can do some experimenting before this. A server can have more than one Moodle on it.
2) I suggest learning how to use the mysql command line to move the database instead of phpmyadmin. phpmyadmin might work, but I have encountered problems with it. The problem grows when your moodle database is larger.
3) You can also have more than one Moodle in you sandbox. Consider trying these steps with your sandbox. It is not clear to me if you have had success yet installing Moodle+theme+plugins on your local Moodle sandbox.

None of what I say is meant to be negative. You are doing a great job learning about Moodle installations.

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 21 July 2021, 5:39 PM

that is a very good point re why wait till Monday

I just have a tight schedule this week with other jobs. But I did have a think and cleared my schedule for tomorrow so I can do a trial run with a diff domain on the live server.
Have not had any success yet! Hoping tomorrow is the day!
Thanks for your advice, will report back

Average of ratings: -

Re: Moodle Install and Migration

by Rick Jerz - Wednesday, 21 July 2021, 9:13 PM

Lauren, to help your overall philosophy, here is a photo of a t-shirt that I got from a MoodleMoot.

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Wednesday, 21 July 2021, 12:45 AM

Do you have an account on existing site that is set to admin level.
IF so ...
Go to existing site now and get a listing of additional plugins/theme's

/admin/plugins.php?updatesonly=0&contribonly=1

In the plugin name column you will see the human name and just below that a behind the scenes name. Example:
Google Meet for Moodle and under that mod_googlemeet
From above you know then that plugin is a mod.

You should also see if those addons have dependencies ... other plugins that are related and must be installed to work.

Wouldn't hurt to check settings of those and record them to a text file on your workstation for checking config once site migrated. (since you mentioned Mac, use Textedit - a text editor.

If you share back that list I'll do you a favor and provide info on latest version of the plugin as well as a link to the zip.

Note: I'd pre-install those plugins via command line prior to pulling the trigger on accessing the site via GUI ... that way the missing from disk will be only those addons former provider had installed (assuming they have not modified core code).

You've said you have the resources needed.
Well, how much memory? how much space?
What is size of the moodledata directory?
What is the size of the DB dump you will be getting?

WHM, CPanel, and finally the OS ... what OS is it? Ubuntu, CentOS, Debian?
Makes a difference.

WHM allows you to set up servers ... with limits. One could have a dedicated DB server
using WHM.   **In your case, you might be allocating on the same server (IP).**
It has tools for settings which rule the roost ... above that of cPanel ... and one should not edit the ini files from command line to bump upwards should you determine that is needed.

So you will have a single server running WHM, cPanel, and of course the moodle?

I would get apache and cert working on new environment first.
Apache port 80 and 443 + whatever cert new provider provides.
In document root, a plain jane index.html file that contains only one line:
'It works'
You can remove that index.html file when doing moodle.

From where will you get moodle 3.9.2?
Current/highest is 3.9.8+
https://download.moodle.org/releases/security/
Also note there are 2 packages ... one for tgz and one for zip.
Zip is larger and probably contains stuff for Windows ... which you don't need.

BTW, the mac download:
https://download.moodle.org/macosx/
Moodle 3.9.8
MOODLE_39_STABLE

8 days 15 hours ago    Moodle4Mac 3.9.8
The 3.9 branch is very old now and is not being improved except for major security and dataloss fixes. The core Moodle team will keep working on them until May 2023.

SourceForge has old versions
https://sourceforge.net/projects/moodle/files/Moodle/stable39/moodle-3.9.2.tgz/download

I would swap 7 and 8.
Do theme last!

Tip: you can force theme via config.php file:
$CFG-theme='boost';

Location of moodledata ... remains to be discovered ... dependent upon using apache's native Linux Standards Base /var/www/ OR you using a user jail ...
/home/[useraccountlogin]/public_html

48 hour ... hmmmm ... put on some coffee.
When you finally tire the first day and need sleep, make sure what you have done is secure enough to leave it in the state it's in.

Biggy ... as you progress forward and you've been able to attain a step, take the time to do a backup of code and DB. You could also do minimal moodledata/filedir/

IF ... IF ... you reach a step that fails and you get white screen of death and cannot figure it out, you don't want to start all over again ... just restore your last site backup.

Above relates to space available.

Opinion ... I would spend this weekend's time installing a 'staging' server that is 3.9.8+ ... staging.sussexbusinessschool.com
and, this might be too much for ya right now, I'd install staging via git.

It has always been recommended to have a 'tinker' site for testing themes/plugins/etc.
before implementing on production server.   Also opinion ... Mac local install isn't same environment and therefore, IMHO, not the thing to do ... you can tinker later.

My 2 cents.

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 21 July 2021, 5:54 PM

"Sos" Ken indeed!

Thank you so much!

I have cleared my schedule for tomorrow to work through this message and have a go at installing the whole thing in a trial domain on the same server. I have a PDF of the additional plugins - I have gone in and made screenshots of most things that I thought maybe useful if they disappear after move. (attached) So yes, have admin access to current moodle site.

fyi: Hubken Group - current provider (so wont need those after mirgartion)

Have logins to download a theme/plugins Remui ( we pay for that separately)

I can make this into txt tomorrow if you need that. Thanks again for "holding my hand" through this.
Anything else from the Admin section overall you think would be good to screenshot, I didn't do every page - there is a lot!

(second screenshot - admin pages I have screenshots of) trying to be prepared

Thanks again, will report back tomorrow!

Attachment Screenshot 2021-07-21 at 10.53.11.png

additional-plugins-php-2021-07-19-10_34_38.pdf

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 21 July 2021, 6:15 PM

Just one more thought that niggling at the back of my head - I saw somewhere that cron jobs need setting up and must run every hour or minute (cant remember fro top of my head) for moodle to work - so something rather important...

which I will need to do on Monday as well before updating DNS and opening the site to students again.

I have not done that (setting up cron jobs) before either - so any advice on this area is also very welcome

Got some info re server below from WHM: (most gobbledygook to me but maybe you can spot something I should tweak or upgrade, prettu sure my server should be OK to handle the load, I don't know how big the actual site is, however...... current provided not really telling me squat apart from - you will get the backups of DT and moodledata monday and then bye bye.

My server is CentOS Linux release 7.9.2009 (Core)

Server Version: Apache/2.4.48 (cPanel) OpenSSL/1.1.1k mod_bwlimited/1.4
Server MPM: prefork
Server Built: Jun 30 2021 17:18:56
Server load: 0.10 0.23 0.20
Total accesses: 1439227 - Total Traffic: 59.2 GB - Total Duration: 850858328
CPU Usage: u9.56 s56.52 cu1929.32 cs1305.77 - .187% CPU load

Server Load 0.138672 (8 CPUs)
Memory Used 26.63% (4,285,544 of 16,094,416)
Swap Used 0.02% (1,012 of 4,194,300)

Total processors: 8
Processors: (x8)
Vendor GenuineIntel
Name Intel(R) Xeon(R) CPU E3-1230 v6 @ 3.50GHz
Speed 3500.000 MHz
Cache 8192 KB

Memory Information:
[ 0.000000] Memory: 4016380k/18612224k available (6916k kernel code, 2008532k absent, 530716k reserved, 4551k data, 1800k init)

System Information:
Linux 80.244.176.38.srvlist.ukfast.net 3.10.0-693.11.6.el7.x86_64 #1 SMP Thu Jan 4 01:06:37 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Physical Disks:
[ 2.369081] sd 0:2:0:0: [sda] 233308160 512-byte logical blocks: (119 GB/111 GiB)
[ 2.369733] sd 0:2:0:0: [sda] Write Protect is off
[ 2.370044] sd 0:2:0:0: [sda] Mode Sense: 1f 00 10 08
[ 2.370062] sd 0:2:0:0: [sda] Write cache: disabled, read cache: disabled, supports DPO and FUA
[ 2.371909] sda: sda1 sda2
[ 2.372469] sd 0:2:0:0: [sda] Attached SCSI disk
[ 3.250723] sd 0:2:0:0: Attached scsi generic sg0 type 0

Current Memory Usage:
total used free shared buff/cache available
Mem: 16094416 2777764 2457980 133760 10858672 12841492
Swap: 4194300 1012 4193288
Total: 20288716 2778776 6651268

Current Disk Usage:
Filesystem Size Used Avail Use% Mounted on
devtmpfs 7.7G 0 7.7G 0% /dev
tmpfs 7.7G 0 7.7G 0% /dev/shm
tmpfs 7.7G 1.1M 7.7G 1% /run
tmpfs 7.7G 0 7.7G 0% /sys/fs/cgroup
/dev/mapper/vg_main-lv_root 104G 41G 58G 42% /
/dev/sda1 976M 144M 766M 16% /boot
tmpfs 1.6G 0 1.6G 0% /run/user/0

Average of ratings: -

Re: Moodle Install and Migration

by Rick Jerz - Wednesday, 21 July 2021, 9:19 PM

I show one way to set up cron in the videos that I shared with you (above.) You should be able to do this from your cPanel.

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Thursday, 22 July 2021, 1:27 AM

Ok, thanks for info ... now some suggestions and commands to use on new server that are more user friendly ...

df -h

disk free and modifier -h means show in human terms.

what is perhaps most important is the / line

*copy* the text of what you see, and paste it in a response here.

Another:

free -h

relates to memory

Ok, the Group thang ... yeah, you're not going to get that code but there will be references to it in DB mdl_config_plugins and DB will have related tables.

What does it do?

A migration such as this is like an iceburg ... what is it with a burg .... 3/4 of the burg is below the surface?

48 hours ... uhhhh ... errrr ... think I'd inquire about amending that and possibly extending - a staging attempt, while that takes time, should give you a better estimate of migration of production site.

Keep your coffee brewer functional and handy!

Reminder ... shortnames in a text file.

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Friday, 23 July 2021, 3:04 AM

Dunno how your new hosting environment handles OS (centos 7) updates.

You might want to sign up:
https://lists.centos.org/mailman/listinfo/centos-announce

You are administering the entire server ... not just apache/mysql and the apps,
but CentOS 7 as well.

Within the last 2 weeks ... critical systemd update ... yesterday a kernel update.

No internet server is install it once and forget it.

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 21 July 2021, 6:37 PM

PS this is fun

I feel like a proper developer on the black screen

thank you!

Attachment Screenshot 2021-07-21 at 11.35.48.png

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Wednesday, 21 July 2021, 10:30 PM

Combo response here ... and a quickie ... have Dr appointment.

Need text file for additional plugins - the short name.
can't highlight/copy and paste the short name.

Warning about copy and paste to command line ... ** must be accurate with highlighting ** don't extend beyond the last character or you add a hidden carriage return (presses enter for you - and that's not good).

But ... for now ... and briefly ...
BigBlueButton requires a BBB server.

Check settings for that plugin on current server and how often that is used.

/admin/modules.php

Activities column ... shows a number ... that's how many instances of the plugin are in the site. Clicking number shows what courses that activity is in.

H5P is part of core now ... not an add-on mod. Not sure how this will be handled by your stock 3.9. Short name in a stock 3.9.highest is mod_h5pactivity. There is a built in addition for getting the additional plugins for it now.

Edwiser RemUI block is tied to theme as is Edwiser course formats

Will repeat ... think it best to worry about/get working core first ... no $ theme and related plugins. Then get the $ addons.

Also .. on current server make one full course backup and download that backup - it's an .mbz file and really not of use on your workstation but will be handy on new server.

Ok, off for appointment! :|

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Thursday, 22 July 2021, 8:23 PM

Hello gentlemen

Thank you again for all the advice!
I have installed 3.9.8+ and got it working in a trial domain:
moodle.lubilabs.uk

I had one warning during install regarding site not being secure (see screenshot)
Everything looked like it should without HTTPS (see screenshot)

Then I went and added cPanel SSL and https redirect and everything now looks like CSS is missing or something.. like basic HTML load will attach screenshot in the next message.

install-warning.pdf

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Thursday, 22 July 2021, 8:27 PM

was about to start celebrating. Maybe a need to update the site location in config.php to https:// ?

or does it need a special SSL Certificate? I only added auto ssl from cPanel (DV) and enabled forse https redirect under "Domains" in cPanel - which is what I normally do for small brochure sites... but maybe for this need something more invoivled?

Thank you for any thoughts!

Lauren

Attachment dashboard-after-https-redirect.png

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Thursday, 22 July 2021, 9:11 PM

Ha, i think I have answered my own question

I have updated the config.php and all is looking fine now.
Moving on to adding theme and trying to lean how to set up cron jobs!

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Thursday, 22 July 2021, 10:34 PM

You did answer your own question - kinda -

https://docs.moodle.org/39/en/Search_and_replace_tool

https://docs.moodle.org/30/en/HTTPS_Search_and_replace

Moodle uses wwwroot variable from config.php to concactate internal urls which are then recorded in row of some tables of the DB.

+ 2 cent suggestions for the test site ...

Since it is a test site, set front page to force login - will not display any categories or courses. Must login to see anything ... no guest login.

In config.php
$CFG->noemailever = true;

IF you do the wise thing (?) and restore a full user backup course from the site you will be migrating. That course should have an H5P (in old site that was an add-on ... in your 3.9.highest it's part of core). And that course should also have a BBB resource in it ... that's assuming the site to migrate actually used those.

Cron job, if restored course had forums or assignments with due dates in it, might attempt notifications to students of that course. IF ... IF ... email is successful, will be confusing to students/teachers/admins that get notices.

Can see you are a 'form' person rather than 'function' - working on theme first.
Comment: it might be the greatest looking moodle ever, but if it doesn't function, what good is it? What of other plugins they are using?

And, what have you learned so far when it comes to migration? That's food for thought.

Progress is good ... just remember a migration + upgrade isn't really the same.

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Rick Jerz - Thursday, 22 July 2021, 11:48 PM

Yep, Lauren, your Moodle looks like a normal new install of Moodle.

You should realize that you have made it! Your moodle could be used to set up courses, add users, and begin using. (Yes, I know that this particular moodle is experimental.) You don't need to add a theme, you could use your Moodle as is. You might want to go to Site Admin | Appearance | Boost and play around with some of the built-in settings.

However, you do need to set up cron. You have been getting lots of ideas. My cron represents only one of many ways to do this, but here is the one line in my cron (which you set up by going into your cPanel|Cron jobs tool.

/usr/local/bin/php /home/myserveruser/public_html/moodle/admin/cli/cron.php >/dev/null 2>&1

You will need to figure out where your php is located. Mine is in /usr/local/bin. And you will need to point to where your cron.php file is located. Yes, you can have a separate cron job for your experimental moodle and for your eventual "production" moodle.

(Continue following Ken's good advice.)

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Tuesday, 27 July 2021, 8:51 PM

Hi guys,

I hope everyone is well?

Well. Yesterday was the day, but... the provider took all day to back up the site and only provided a link to download the DT and moodledata this morning. In the meantime the site is in Maintenance mode and really cannot stay that way for much longer...

the DT is small enough 237MB but the moodledata folder is 80GB
I tried downloading it, but it looks like it will take me 3 days to just download it, uploading it to the server will probably take even longer....

So, we have reopened the site again on the old providers's server, but we have until the end of the week in contract - so the new plan is to reduce the site of the site in the next 2 days as much as possible so that I can download and upload the moodledata in a reasonable amount of time.

--------------------------

I had this from the tech guy there:

" the vast majority of your data is taken up by course backups within the moodledata/repository/backup/ folder and is a total of 52GB"

they are not explaining to me whether these are needed or not for operation and my moodle knowledge as you know.... beginnner!

So any advice you can give me on reducing the size of moodledata directory, without obviously removing anything that is required for operation and is used by existing courses, students etc.

Thanks guys!
Really appreciate any advice you can throw my way on this.

I dont have access to the server only admin logins to the moodle site, so I can tidy up from there and hopefully ask them to remove stuff before making the next backup - what stuff to remove... please help me understand..

Thank you thank you thank you!

Lauren

Average of ratings: -

Re: Moodle Install and Migration

by Leon Stringer - Tuesday, 27 July 2021, 9:45 PM

Can you tell us how the Moodledata is being provided to you? One big file, like a Zip file or tarball (.tar.gz file)? Or individual files? If it's individual files then perhaps you can use Rsync to get these.

Also, can you tell us how the network access to the backups is being provided? SSH/SCP? Web link? SFTP?

80 GB is large but I wouldn't expect it to take days to transfer this. Is it possible to copy it directly from the provider's server to the new server, instead of downloading it and uploading it? That way you can leave it copying overnight and hopefully both servers have faster connections.

I would recommend getting the whole 80 GB copy they're providing, and keep a copy of this. Because obviously it's easier to recover something that you need if you've got a copy, but if you've deleted something to save space then it's gone for good.

That said, I think you could probably skip anything in moodledata/repository/backup and Moodle would work. The folder moodledata/filedir is the most important one.

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Tuesday, 27 July 2021, 10:08 PM

Hi Leon,

Thanks so much for responding!
The backup is provided via a download link from our moodle site:
Like this:
https://moodle.sussexbusinessschool.com/out
Then I log in as admin and see the files available to download (see screenshot) - so its all in one file.

They will not give me access to their server I have asked.. they just ignore the question or say something about proprietory bla bla

I have looked in the Backups settings and we had 30 days to keep backups for and back up everyday, so assume these 52GB are just backups going back a month.

I have just changed this to 1 day instead of 30 days - i imagine this will dramatically reduce the moodledata/repository/backup (second screenshot)

I have pretty fast internet... well for UK

140Mbps / 35Mbps

We have now opened the site back up and trying to get another backup from them on Thursday evening. Was hoping to delete some old students and redundant courses and hopefully our backup will be more like 35GB on Thu.

I will need to download it unzip and then upload via FTP? (binary transfer mode...?)

If there is a better way to upload to the server ( I have full access to mine) once I have this .gz downloaded do let me know

Thank you!

Attachment Screenshot 2021-07-27 at 14.56.40.png

Attachment Screenshot 2021-07-27 at 15.03.10.png

Average of ratings: -

Re: Moodle Install and Migration

by Leon Stringer - Tuesday, 27 July 2021, 10:43 PM

Re download link: it's frustrating that you can't transfer the files directly to the target server but that's what we're stuck with.

I would upload moodledata.tar.gz to the target server and then extract the files there.

To upload this file you can use FTP in binary mode if that's what you're used to. Ideally instead use SCP (secure copy) which should work if you have SSH access. If you're using a Mac or Linux workstation then SCP is should already be there. If you're using Microsoft Windows then you can install PuTTY to get PSCP which does the same thing. So on your workstation at the command line (Command Prompt window if using Windows):

cd DOWNLOAD_FOLDER
scp -C moodledata.tar.gz sbs2108x2@<HOSTADDRESS>:/home/sbs2108x2

Where DOWNLOAD_FOLDER is the folder containing the downloaded moodledata.tar.gz and HOSTADDRESS is the domain name or IP address of the target host. Use pscp instead of scp if that's what you have.

When the upload is complete, on the target server run:

cd /home/sbs2108x2
tar -tzf moodledata.tar.gz | more

which will list the files contained and, importantly, the folder structure so you'll know if it contains the moodledata folder (e.g. course files are in moodledata/filedir), or the files and folders within the moodledata folder (e.g. course files are in filedir). You can then extract the files with:

tar -xzf moodledata.tar.gz

It looks like the backup setting you have changed is just the log file setting, not the backups themselves. You may want to change the following:

Screenshot showing 'Maximum number of backups kept' and 'Delete backups older than' settings

But, these settings may not apply to backups in the moodledata/repository/backup folder. I'd need to look into how the old backups can be deleted from this location, hopefully someone can jump in with an answer.

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Wednesday, 28 July 2021, 2:25 AM

Ok, further wget testing does show one must login to acquire those files.

So - plan B - use curl

So from the new server via shell/terminal and located in the xfer directory you created to hold the transfer, try one of the following or both:

curl https://username:passwd@server1.cyberciti.biz/file/path/data.tar.gz
curl -u Username:Password https://server1.cyberciti.biz/file/path/data.tar.gz

Obviously, you replace username password server path filename in above examples.

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Thursday, 29 July 2021, 1:25 AM

Use of curl to test download ... have a moodle39 code directory (works) ... have an old DB dump and a tar ball of old moodle code.   Copied both files to root of moodle39 code directory, then did a curl test on my mac to acquire
those files.

Test results successful.   You have a Mac so you can do same test from your terminal.

Kens-MacBook-Pro-2:~ kentask$ pwd
/Users/kentask

Kens-MacBook-Pro-2:~ kentask$ cd Desktop

Kens-MacBook-Pro-2:Desktop kentask$ mkdir tdl - make a directory for 'test download' called tdl

Kens-MacBook-Pro-2:Desktop kentask$ cd tdl

Issued curl commands

Kens-MacBook-Pro-2:tdl kentask$ curl https://sos.sosoftexas.org/moodle39/moodle394+-db-20210218134330.sql --output moodle394+-db-20210218134330.sql
% Total    % Received % Xferd Average Speed   Time    Time     Time Current
                                 Dload Upload   Total   Spent    Left Speed
100 25.2M 100 25.2M    0     0 19.4M      0 0:00:01 0:00:01 --:--:-- 19.4M

Kens-MacBook-Pro-2:tdl kentask$ curl https://sos.sosoftexas.org/moodle39/moodle-code-394+-20210218134308.tar --output moodle-code-394+-20210218134308.tar
% Total    % Received % Xferd Average Speed   Time    Time     Time Current
                                 Dload Upload   Total   Spent    Left Speed
100 693M 100 693M    0     0 27.6M      0 0:00:25 0:00:25 --:--:-- 27.8M

IF it works with a Mac, should work with your new hosting CLI and the curl that is on that new hosting.

Basic command:

curl urltofile --output nameofilewithextensiontosave (the .sql and .tar.gz)

*Note for others who see this ... the test files are no longer in moodle39 on my server. So go play somewhere else!!!

Also, if you play too much your ip address will be added to the drop zone and you will never see that server again from that IP. You have been warned!

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Thursday, 29 July 2021, 3:01 AM

Follow up ...

Best if one could go server to server when migrating, but if one cannot do that, down (to your Mac) ... *** clean up *** ... recreate a smaller moodledata.tar.gz by cleaning up moodledata/ leaving filedir and repository/backup/ with only one copy of course backups. Mac can re-create the reduced size moodledata to a moodledata.tar.gz and then upload that to new server via whatever works ... scp as root user comes to mind. You are in full control of your new server so that should not be a problem.

Just one more thing ... once you put the original site in maintenance mode, the clock is running ... depending upon factors unknown yet ... that 48 hour window hopefully enough ... if things go well and you are not in need of just in time support (these forums are not good for just in time support) ... unfortunately, you in UK me in USA (dunno where Leon is) means just in time support difficult. Am too old to pull all-nighters anymore ... we old people need ZZZZZ's.

If it takes longer than 48 hours so be it. Your customer must understand that and accept that.

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Rick Jerz - Tuesday, 27 July 2021, 10:03 PM

Ken, Leon, and Visvanath are much better at these kinds of questions than I am, so I hope that they correct any misguidance that I provide.

It seems to me that you can safely delete course backups since your backups are backups of courses that still exist in your Moodle. Right? However, if your course backups were done, and then you did something to the real courses, perhaps the backups would be useful. Also, if you were doing course backups for the purpose of trying to preserve "changes" then your course backups might be useful. So, ask yourself, "What purpose are your course backups serving?"

I don't do automated course backups, so this is why I am not the best at guiding you. Every semester, however, I do make a course backup and then restore the backup to a new course for the next semester. After I have restored a course, I delete the backup file (manually.)

I think that Moodle eventually removes the deleted backup file. I can't recall if there is a setting that instructs Moodle to do this, or if there is a setting that tells Moodle how often to remove the deleted backups. I just let it happen.

80GB would be a lot for me. If you get it down to 30GB, much better. As a point of reference, my moodle currently has about 50 courses and its moodledata folder (compressed to a gz file) is about 10GB (so, about .2GB per course) My MySQL backup is about 2.5GB. Yes, I do move these backups around almost every day, to a secondary web server, and down to my Mac. It takes about 15-30 minutes to download these files to my Mac via SSH (I download the moodledata.gz, the sql dump, and a moodle.gz application backup.) Yes, a little longer if I were to upload them. I am on a fast Internet connection. My estimate is that an 80GB download would happen in less than 4 hours. Can you download these files from a faster Internet?

If you had a second server, it might be that moving the backups from server to server (not down to your computer) would be faster.

(I see that Leon just posted some guidance, too.)

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Tuesday, 27 July 2021, 10:22 PM

Thanks Rick,

That's interesting that your DB (2.5GB) is considerably larger than ours (237MB). I wonder what that actually means in real terms...

We have around 200 courses, small and grouped as units of larger courses - so actually, about 20 courses but individual units are set up as individual courses if you catch my drift

I think that backup is huge because we had automatic backups set up for every day of the week and keeping them for 30 days. All default settings in Moodle - looks like.

I hope updating this will reduce the size, and I will certainly look to learn more about backups and the best way to do these.

Thank you for your info!

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Tuesday, 27 July 2021, 10:28 PM

Also guys!

Can you help me understand what is backed up in this course backup?
is this just the info we add as course content or is this also the student data? - assignments, assessments chats with tutors etc
maybe there is a separate place for assignments etc?

Average of ratings: -

Re: Moodle Install and Migration

by Leon Stringer - Tuesday, 27 July 2021, 10:53 PM

What is backed up? It depends what is checked under Site administration ▸ Courses ▸ Backups ▸ Automated backup setup under heading Automated backup settings. But it could be everything needed to re-create the course including users and their data.

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 28 July 2021, 7:56 PM

Pretty much everything is selected, looks like all the default settings. So everything is being backed up I imagine (see screenshot)
I have a new problem - asked the provider if they can delete the backups before making the second backup - they said no. (Screenshot 2)
The only info they give me is that that depository folder is 52GB
I have updated the settings for backups, but not sure whether this will delete the backups they have for every day going back 30 days and only leave 1 latest backup. In theory that should make that folder around 2GB... Looks like they are not offering any help at all with this.
How do I ensure that I have deleted this and verify before tomorrow pm when they do another backup they charge us £360 every time, so we really don't want to pay for a third one!

Thanks Leon, I really appreciate your time man!

Attachment Screenshot 2021-07-28 at 11.21.21-1.jpg

Attachment Screenshot 2021-07-28 at 12.43.48.png

Average of ratings: -

Re: Moodle Install and Migration

by Leon Stringer - Wednesday, 28 July 2021, 10:55 PM

I don't think you can delete those files from within Moodle.

By default Moodle puts automatic course backups in its managed file store, the filedir subdirectory in Moodledata. Such backups could be deleted manually from within Moodle.

But this site has been configured to put the course backups in a different directory (Automated backup storage: Specified directory for automated backups and Save to: /var/www/moodledata/repository/backup). You cannot manage these files in Moodle as far as I'm aware, even though this directory is in Moodledata.

But it looks to me like this was configured by the provider, in your screenshot these settings have Defined in config.php by them. If the provider is saying it's your responsibility to delete them then perhaps you should ask how you should do that given the way the site is configured. (Depending on the history of the site there's a possibility that the above settings were set by a previous provider).

If anyone else has any ideas how to delete these course backup files hopefully they'll jump in.

Average of ratings: -

Re: Moodle Install and Migration

by Rick Jerz - Tuesday, 27 July 2021, 11:29 PM

My guess is that my DB is larger than yours because my courses have many more "interactive" components to them. For example, I allow (encourage) students to repeat quizzes. And I tend to use a lot of forums. I use "assignments" to a much smaller extent, so students are not uploading much (which goes into moodledata.) I also never upload video or audios into my Moodle, and my content pages are external, too (yet small when compared to video files.) I don't use automated backups, and I always clean my old backup/restore course files. I am careful about controlling the size of moodledata, since I pay for my own server and don't want to buy big storage space.

I think by now you realize that the DB goes along with the moodledata folder. One, by itself, does you little good (when restored.)

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Tuesday, 27 July 2021, 10:25 PM

My 2 cents:

Course backups are not required for a migration, however, they are the 'plan B'
for new site because course restore process does allow deselecting resources
in a course when restoring.

Ask current provider to make 2 tar balls ... one for moodledata/filedir/ only
and one for moodledata/repository/backups/

On current server use ssh and make sure you are located on a partition that has enough free space to acquire the 2 tar balls. Then use command line only
wget to acquire the tar balls/files.

Command looks like:
wget https://site/location/filename.ext

**Must point to the file.** Three wget commands for each backup they provide.

That does the transfer server to server ... not download then upload.

Original site should be left in maintenance mode ... no new activity by teachers or students for however long the migration and setup of new server takes.

'SoS', Ken

Average of ratings: Useful (1)

Re: Moodle Install and Migration

by Ken Task - Tuesday, 27 July 2021, 10:44 PM

And a follow up question ... I see a schema.sql file. Might ask current provider how they made the db dump - what command they used.

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Tuesday, 27 July 2021, 10:47 PM

OOOh Ken, this looks promising!

I doubt they will do two files, they are being total .....

Assuming I can only get one file of the whole moodledata directory could I in theory got to Terminal window in my hosting account and do something like:

cd moodledata

this should get me into the right directory... right?

then what you said:

wget https://moodle.sussexbusinessschool.com/out/moodledata.tar.gz

This sounds too simple

SOS!

Average of ratings: -

Re: Moodle Install and Migration

by Leon Stringer - Tuesday, 27 July 2021, 11:15 PM

The challenge is that you need to log in to /out get access to the download link for moodledata.tar.gz, etc., and presumably (hopefully!) their server is checking your browser has logged in before allowing the download.

Wget won't do that by default but you might be able to get this to work by sending login information first. Something like:

1. Send the login credentials and save any cookies set by the provider's server:

wget -O- --keep-session-cookies --save-cookies cookies.txt --post-data 'fm_usr=MYUSER&fm_pwd=MYPASS' https://moodle.sussexbusinessschool.com/out/

2. Try to download the file including the cookies as part of the request:

wget --load-cookies cookies.txt https://moodle.sussexbusinessschool.com/out/moodledata.tar.gz

The above link is possibly wrong, you'll need to copy the link from the File Manager page, possibly from the blue button on the right hand side:

Ccreenshot of the File Manager at https://.../out.

This might not work of course. (Steps based on this page).

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 28 July 2021, 5:42 PM

Hi Leon,
Thanks so much I want to try this as I have the prev backup DT sitting there and a small file.

I have done the Step 1 command as you said:

wget -O- --keep-session-cookies --save-cookies cookies.txt --post-data 'fm_usr=MYUSER&fm_pwd=MYPASS' https://moodle.sussexbusinessschool.com/out/

it returned this:

(screenshot 2)

I think that gotta mean it worked

When i click on the blue copy icon inside the file manager i get this (screenshot 1)

Is that something I need to include in my next command? to try and copy that DT file to test the process?

Also where should I put it, does it matter? I can delete via FTP after right? Can just plop it into the home folder by default?

Like you said the link prob wrong, does the screenshot help to clarify the command for step 2 I shoudl try?

Sorry I have a good brain but zero experience so just following your lead

Would be good to verify the process works for the real thing on Friday when I get the new backup hopefully no bigger that 30GB

Thank you so much for your guidance! I already learned a lot! if you have a paypal account I can send you a beer or cookies LOL whichever is your prefference

I already created the /moodledata directory one level up from www and did the right permissions (you helped!)

Have also uploaded the correct version of moodle 3.9.8 (same as they are running) and updated the config.php with DT credentials and locations of moodledata etc. Also added the theme files that they were running - so I really think I am prepared to import DT and moodledata and everything should! work...

I will only get a single moodledata file - is it possible to tell the terminal to plop it in the right place and unzip there?

the path to my moodledata folder is home/sbs2108x2/moodledata

it is currently empty obv

Thanks Leon!

Attachment Screenshot 2021-07-28 at 10.06.02.png

Attachment Screenshot 2021-07-28 at 10.21.47-1.jpg

Average of ratings: -

Re: Moodle Install and Migration

by Leon Stringer - Wednesday, 28 July 2021, 6:09 PM

For that second wget command we're looking for the URL that the browser would use to download moodledata.tar.gz. So if you tried the download again in your browser, the one you said looked like it would take three days, then hopefully that's the same link and you can copy that. Then cancel the download in the browser and try it with wget:

wget --load-cookies cookies.txt https://<URL_COPIED_FROM_BROWSER>

Re extracting the Moodledata files, this is what I meant in this reply:

When the upload is complete, on the target server run:

cd /home/sbs2108x2
tar -tzf moodledata.tar.gz | more

which will list the files contained and, importantly, the folder structure so you'll know if it contains the moodledata folder (e.g. course files are in moodledata/filedir), or the files and folders within the moodledata folder (e.g. course files are in filedir). You can then extract the files with:

tar -xzf moodledata.tar.gz

You need to see what's in moodledata.tar.gz (with tar -tgz …) to find out where to extract the contents. It probably contains the moodledata directory in which case you would extract the files from the /home/sbs2108x2 directory. When moodledata.tar.gz has been transferred to the target server run that command and it may be more obvious what I mean. If not then share the first few lines out output from that command and we should be able to assist.

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 28 July 2021, 8:51 PM

I am not sure that worked..

I did this:

wget --load-cookies cookies.txt https://moodle.sussexbusinessschool.com/out/index.php?p=&dl=schema.sql

but maybe something went squiffy (technical term!) because it did get something but not the schema.sql see screenshot, i think it didnt like the = sign and anything after...

The link for the main file is also same structure:

https://moodle.sussexbusinessschool.com/out/index.php?p=&dl=moodledata.tar.gz

maybe I need to undate this somehow?

Attachment Screenshot 2021-07-28 at 13.49.49.png

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 28 July 2021, 8:57 PM

tried this also...

Attachment Screenshot 2021-07-28 at 13.56.37.png

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 28 July 2021, 9:00 PM

PPS Thanks for the info again re extracting files. I think I understand

Just need to make sure I can get the files and whether I can clear the backups myself without access to the server only via the moodle admin.

*sweaty face emoji!

Average of ratings: -

Re: Moodle Install and Migration

by Leon Stringer - Wednesday, 28 July 2021, 9:33 PM

Try putting the URL inside single quotes, i.e.:

wget --load-cookies cookies.txt 'https://moodle.sussexbusinessschool.com/out/index.php?p=&dl=schema.sql'

or:

wget --load-cookies cookies.txt 'https://moodle.sussexbusinessschool.com/out/index.php?p=&dl=moodledata.tar.gz'

The '&' in the URL is confusing things.

You might have to re-run the first wget with the username and password again if the login has timed out.

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 28 July 2021, 11:21 PM

cool, trying that now!
will report

you said this before:
But, these settings may not apply to backups in the moodledata/repository/backup folder. I'd need to look into how the old backups can be deleted from this location, hopefully someone can jump in with an answer.
--- nobody chimed in yet - and host refusing to delete anything... any ideas?
THXL

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 28 July 2021, 11:28 PM

not sure that worked... it saved the index file i think... not the dt file:

maybe I could ask for a direct link to download without &= etc in it?

not holding my breath though

Attachment Screenshot 2021-07-28 at 16.24.42.png

Attachment Screenshot 2021-07-28 at 16.27.33.png

Average of ratings: -

Re: Moodle Install and Migration

by Leon Stringer - Wednesday, 28 July 2021, 11:47 PM

Maybe https://moodle.sussexbusinessschool.com/out/index.php?p=&dl=schema.sql is the wrong link, maybe it's a page that links to the download link.

You could check again in your browser to double-check the URL that it's actually downloading for schema.sql. This may not actually be the URL in the address bar. Because this File Manager interface is some additional system provided by the current provider – i.e. not Moodle – it's difficult to give pointers as to what you might be looking for.

Maybe the link is in the body part of that file in your screenshot, in the second half of the file. Or maybe there's an error in there which may help.

Or maybe the provider's interface is structured in a way that won't work with wget.

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Thursday, 29 July 2021, 12:44 AM

In looking at what the file has in it, it appears to be a login page! :\

You showed some panel ... cPanel? Could you make a copy of schema.sql file and move it to 'code root' ... where one sees config.php?

IF so, one could try https://site/schema.sql directly in browser to see if one can access that without being logged on. Browsers should not know what to do with a .sql file so hopefully you would be prompted to save the file.

IF that worked, a simple wget should be able to acquire the file ... but Leon is correct ... web service may have a 'bad bots' kinda thing configured and has 'wget' agent defined as a 'bad bot' thus forcing login - thus you get the login page downloaded.

That .sql file is for DB import on new server. It's a text file and much smaller than the .tar.gz. That .sql file one could upload to new server so that you could use it in a command line import to new DB on new hosting.

***Listen*** ... can't hurt to try the curl stuff. Worse that can happen ... it fails also ... won't affect the site ... just can't download what you need to xfer to server.

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Wednesday, 28 July 2021, 12:13 AM

You can ask, right? Ya know they might be only too glad to rid themselves of a pesky customer! If first level of help desk won't ... escalate the request to an 'engineer' level of support. Might be time to get 'nasty' - in a 'polite way'!

Leon is correct in that the wget command might need some extra parameters, but I just tried it (if it started to transfer plan was to cancel the command) and got:

HTTP request sent, awaiting response... 404 Not Found
2021-07-27 10:49:13 ERROR 404: Not Found.

So it appears to have connected ok (without additional parameters to wget) but could not find the file. The /out/ directory?

To test, I used https://server/mod/upgrade.txt ... a file known to be present in moodle code ... and my wget command pointed to /mod/upgrade.txt did transfer:

HTTP request sent, awaiting response... 200 OK
Length: 14525 (14K) [text/plain]
Saving to: ‘upgrade.txt’

upgrade.txt 100%[===================>] 14.18K --.-KB/s in 0.1s

2021-07-27 10:56:39 (110 KB/s) - ‘upgrade.txt’ saved [14525/14525]

So am guessing, if wget is given a proper/allowable path to the .tar.gz and .sql file those will transfer.

About your cd moodledata.

First, your current moodledata is for your fresh install of 3.9.highest ... so the new site won't have the same moodledata (depending upon how configured).

Something like cd /path/to/real/fullyqualfieddomainnameofxferredsite/moodledata would work but that doesn't exist yet.

So, previously, I ask you to run from terminal on your new server:

df -h

and look for the partition with most space or enough space free to xfer the files. Remember, eventually, you will ungzip the .tar.gz's which are compressed so the moodledata directory location of the xferred domain/site will have to have enough free space to handle that as well.

So suggestion ... since you have total access to that drive.

From terminal .. pwd

if enough room there ..

mkdir xfer

cd xfer

use wget command that works at that location. Files will be acquired and saved in the xfer directory.

'SoS', Ken

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 28 July 2021, 5:48 PM

Thanks Ken, processing multiple bits of advice here from you guys - all really helpful and forging a path of my own somewhere in the middle

I think I have the right version for this. I have3.9.8 which is what they are running there.

Hopefully, all I need is to import the DT and moodledata and everything will work... have config.php ready and updated already.

Just need to get this moodledata in a reasonable amount of downtime

Thanks again, guys!

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Wednesday, 28 July 2021, 9:39 PM

Hi again Ken,

I saw a discussion you have been in some 10 years ago almost - https://moodle.org/mod/forum/discuss.php?d=219826
"How to I bulk delete Automated backups".

It seems you may be able to help me here... maybe? Basically there are 52GB of automated backups in the moodledata/repository/backup folder, and the current supplier refuses to delete them before making a backup and says I should do this myself.

Is this possible without access to their server? via Moodle admin?
From that discussion - it seems that by changing the amount of backups to keep does not actully clear them. But that was 10 years ago.. Maybe now it does? If I have updated all the setting to only keep one backup - will it delete the old stuff?

The whole point of doing a round 2 with these guys (£360 each time) was to get rid of the backups and have a much more manageable size of moodledata backup - but after charging me the second 360 they now advised that it is up to me to delete anything and they do not do this.

I am worried I will get the second backup exactly the same as last one 80GB

What are your thoughts on this?
If I can only access via front end admin can i delete stuff? we have 200 courses and 30 days worth of backups.... eeek

SOS!

Average of ratings: -

Re: Moodle Install and Migration

by Ken Task - Thursday, 29 July 2021, 12:04 AM

10 years ... has it been that long??!!!

That discussion was about how to recover disk space because they were out of space and in danger of corrupting the DB.

It involved direct DB queries of moodle db mdl_files table to find the contenthash value, then using command line in moodledata/filedir/ to find the file (named with contenthash value), manually erasing that file (rm command), then going back to same table and removing the row that referenced it.

You don't have access to mysql client nor cli on current host.

Automated backups are sent to moodledata/repository/ named backup. In there you will see backup filenames that are humanly recognizable. If they have 'nu' in their names, it's a no user backup. If no 'nu' in filenames, they are full course backups containing teacher/student users and whatever they did in the course up to the date of that backup.

Moodle Admin interface has no direct way to clean up a file system repository. But one can, IF one 'pretends' to want to 'restore' a course. Note that is 'pretend'. Once clicking restore, select the 'backup' file system resource as to where to find the backup to restore. Once the screen shows all those backup files, one can select (right click?) and choose the option to delete the file. Select next file ... choose option to delete ... repeat until you have only one copy of the 400 backups. Then save that file area. Cancel the restore process.

That removes them from moodledata/repository/backup/ *but* 3.9 has recyclebin ... so the removal has actually led to the creation of more backups tagged as being in recyclebin ... those are moved to trashdir when the cron jobs/scheduled task finally get round to them - total time about 5 days.

To be able to remove recyclebin backups one has to set recyclebin to show all the time in course admin menu so one can click there to see what's in recyclebin ... that will allow immediate deletion ... well, not really ... sends that recyclebin backup to trashdir ... and it sits for 4 days until cron job sees that file is no longer linked anywhere in system and finally deletes the file.

So what one appears to gain space wise when viewing courses and that file system repo has really only moved to trashdir ... in affect, no gain in saving or making that moodledata backup to .tar.gz smaller.

400 courses is gonna take a very long time ... probably right up to the time they are making a tar gz of moodledata.

It's for above that I suggested you ask them to make a .tar.gz of only moodledata/filedir/ That's the only directory you must absolutely have to restore your site on new hosting.

And them saying they can't make a .tar.gz of just moodledata/filedir/ is (to put it nicely) 'poppycock'!

For folks who might be reading this saga, take note of where it's hosted and never, never, never host your moodle there.

'SoS', Ken

Average of ratings: Useful (1)

Re: Moodle Install and Migration

by Lauren Land - Thursday, 29 July 2021, 2:50 PM

Ahahaha Ken, you made me laugh this morning. Thank you! for the bold addendum to your message

I am loosing sleep over this, up at 5 today after dreaming about moodle and waking up annoyed at myself for doing so

In case this is not clear where it is hosted for those reading this saga I can clarify:

https://www.hubkengroup.com/

Off to make some coffee

Ps Ken - I have asked my boss who happens to know the owner of this company to politely explain that her best employee is about to lose her shit (not like me at all) due to the worst customer service and unwillingness to help encountered... like ever.

He promised to investigate today, so there is a possibility I could actually ask for filedir on its own. Again not holding my breath though.. also I could ask for a real download link that could work with wget...

like really, that would be so much easier, right? LOL

I Will report later. Thank you for hanging around still, this thread is a monster. I am getting glost referring to stuff I remember was said

Average of ratings: -

Re: Moodle Install and Migration

by Lauren Land - Monday, 2 August 2021, 11:47 PM

Hello everyone, thought I would update you all on the resolution of this saga...

My boss could not justify the downtime that will be needed to migrate the site so we had to renew for another year, reluctantly...

I am now going to build a new Moodle site from scratch and without the pressure of deadlines etc in a different domain and we can slowly build it and optimise it properly before finally leaving those lovely people once I build the new site and migrate courses / students slowly over. I am sure its not as easy as it seems... but thats the plan.

I have learned so much, thank you all and it will come in very handy with my new site. I will be around asking more questions soon - need a break from moodle for a couple of weeks after all this!

Thanks again everyone, and hopefully I can start enjoying working with Moodle when there are no deadlines and freedom to learn and develop as I please

Lauren

Average of ratings: -

Re: Moodle Install and Migration

by Rick Jerz - Tuesday, 3 August 2021, 2:57 AM

Thanks for the update, Lauren. By next year, you will be the boss and then you can make the correct decisions. Good work. And good plans for next year.

When I do my experimenting, I try to buy only one month from the server company. Then I hit it hard experimenting for a month.

Average of ratings: -