Upgrading Moodle on a Shared Server/Hosted Service Provider
Moodle upgrade steps from 1.9.5 to 2.4.1
These notes are provided in an effort to make the upgrade process easier for others having to go through the same process as I did. Several people over time helped on the forum site and in particular Ken Task was an incredible help! I kept notes on the process for reference and to then share in the forum to help others as I had wished someone else had done this. I’m sharing this info “in the spirit of sharing” (just like Ken says).
**Very important note: these steps worked for me and I can't guarantee they will work for you; and likewise these notes are focused specifically for people in my situation that are on a Shared Server/Hosted Service Provider, otherwise there are probably more efficient steps for people who are own their own local server that they own and have access to a Linux command prompt using putty or something similar. For the rest of us that are on a shared server provided by a hosting service, we are greatly limited on what we can do and what we have access to - these notes are intended for that audience.
These upgrade notes started when I was upgrading from 1.9.5 to 2.4.1. I was behind on my versions because I am on a shared server with a hosted service provider and therefore had many restrictions (mainly based on the versions of PHP and MySQL that I could run) so I was ultimately forced to move to a new hosting provider that supported the latest versions. I was on Network Solutions and couldn’t upgrade, looked into Godaddy and had the same restrictions on a hosted server. I opened a new account with JustHost.com and they offer the latest versions of PHP and MySQL and I was able to upgrade to the latest version of Moodle which at the time of this writing was 2.4.1. Justhost.com has great service and support and I recommend them as a service provider.
These notes are unique to this situation of being on a shared server with a hosted service provider and due to the restrictions you will run into with a hosted service provider you can not do a lot of the steps people talk about in other blog posts like using git, ssh and command level steps. The process steps below will get you upgraded in this restricted environment.
The versions of Moodle that I upgraded to are as follows in this order:
1.9.5 (the version I was already on)
The upgrade process was extremely frustrating (as you can see from posts for so many people) and there didn’t appear to be any detailed list of what to actually do - most notes were missing steps or didn’t apply to being in the situation of being on a hosted server which is so much more restrictive. These notes may not work for everyone and are probably not perfect but ideally it will really help someone else out that is going through the headaches I went through. The important steps were keeping copies of the old installations for reference, doing a MySQL dump to move the database and importing that sql file into an empty database. I upgraded to 1.9.19 to 2.2.7 to 2.3.4 to 2.4.1 and I used these notes for each upgrade and updated them when necessary so ideally they are pretty accurate and should work for you. Note that if you are upgrading from 1.9.5 like I was, then you need to upgrade to 1.9.19 first as there were significant changes to moodle that are needed to jump to 2.x.
I used a couple of tools to complete this process:
- FileZilla - to move the major directory structures
- moodle: all versions are uploaded using FileZilla
- moodledata when transferring from the old server to the new one (only this initial time).
- phpMyAdmin - used to perform MySQL dumps and imports
- Hosted Service Provider’s “File Manager” - this is useful when moving files around on the server itself and especially when you want to create a “copy” of a whole directory structure as a backup.
- download moodle zip file for the version you want to install
- uncompress zip file locally
- upload to new server (public_html directory - should have 755 permissions recursive)
- copy moodledata directory from your old serer to your local computer (I copy the moodledata directory to another name to have a backup. I use the version number in the name like moodledata1_9_19)
- rename the actual moodle directory as you are upgrading from one version to another. This way you have access to old files if needed like the config.php file (this is a very important file you need during upgrades; don't loose it). I rename the "moodle" directory to something like “moodle1_9_19" and then FTP the new moodle directory structure onto the new server.
- upload moodledata directory to root location (one level above public_html with permissions set to 777)
- install book plugin if you use it and you are on a version prior to 2.3
> download book plugin
>> pay attention, there are different book versions out there so make sure you install the right one
> unzip locally
> ftp to new server installation in /moodle/mod/
- make sure the php.ini file contains “register_globals = Off”
- old server:
- use phpMyAdmin to perform a MySQL Dump (I used instructions found in WordPress http://codex.wordpress.org/Backing_Up_Your_Database in the "Using phpMyAdmin" section). This produces a .sql file
- edit the .sql file to comment out the first create statement which is one to create the database but the database will already exist.
- if necessary, change the "use <database name>" statement to reference the new database name on the new server.
- if you are moving your site to a new server (hosting provider) and have not moved your DNS yet as you want your existing site to still run while you are setting up the new site (which was my scenario), then edit the .sql file using a tool like “Notepad++” (needs to be an editor that can handle large files) and perform a search and replace looking for your old site name/DNS (like “www.abcorp.com”) and replace it with the new one which is probably an IP Address and sub-directory name (like “126.96.36.199/newsubname”). If using Notepad++ then use the “count” feature in the search and replace window to see the number of items. After you make a change, do a “count” on some other deviations of your domain name to make sure you got all the “URL” references (don’t change any email addresses).
> new server:
- create a new empty database, make sure MySQL setting is unicode
> write down the database name, username and password for the admin account (needed for the moodle install)
- using phpMyAdmin (MySQL)
- click databases
- click on database name
- click on operations tab
- go to collation group and set to utf8-unicode_ci
>>note that there are two levels of this setting, a top level and bottom level, make sure the bottom level is set
- Make sure php is using "PHP 5.4 (single php.ini)" if you have that option
- change php.ini (most likely in /cgi_bin/php.ini) setting to "register_globals = off"
- database / MySQL
>> if you had issues previously and are running through this process again, then make sure the database is empty. if there are any existing tables, then drop them all otherwise you'll end up with conflicts when the SQL dump is imported again.
>>> in dropping existing tables, I did this using Notepad++. Using the left sidbar in phpMyAdmin I copied all the tables into Notepadd++ (pay attention to whether you have more than one page of tables as there could be more tables on a page 2), used column editing (Alt+Shift+arrow) to select "structure" and delete it. Then did the column highlight again and added "Drop table " to the front of every line. Then used "search > replace" and put $ for the "find what" and put ; for "replace with" (this will put a semicolon at the end of every sql statement) and selected "regular expression" for search mode. You should now have a list of Drop table statements for each table. Now copy and paste this entire list into the SQL tab in phpMyAdmin and execute it. This will drop all the tables. Do a reload on the database and all the tables should disappear and your database is empty.
Import MySQL dump from previous database using the sql file (this will set it up just like the previous database that will now be upgraded but this process keeps and old version for you to fall back to if necessary):
- go to the Import tab and specify your sql file to recreate tables which also brings in all the data for the courses.
**With a new installation, check to see if there is a “config.php” file in your moodle directory. If there is one then rename it because the installation will not run if there is a config.php file present already.
- in a new browser tab, access your moodle installation which will kick off the installation/upgrade
basically something like <url>/moodle/install.php
> for me I basically take all the defaults; I’m not doing anything fancy with my classes so all the defaults work.
- Password salting was introduced in 1.9.19, so if you are going from this version to a 2.x version (I jumped from 1.9.19 to 2.2.7 based on a recommendation) then you have to carry over the salting from the 1.9.19 config file. Only do this step when you are doing the upgrade to 2.2.7 (the first 2.x upgrade version). Do the following:
- Open the 1.9.19 config file and copy the $CFG->passwordsaltmain line = ‘xxxxx’;
- Paste this into a text editor like notepad and edit it to use “passwordsaltalt1” instead, so it would look like:
$CFG->passwordsaltalt1 = 'xxxx';
You can keep up to 20 previous salts. Note in my example the “xxxx” would really be the actual encoding
- Open the config.php file in the new installation
- Keep the existing $CFG->passwordsaltmain line but now paste in the $CFG->passwordsaltalt1 = 'xxxx'; line
This is very important, without the previous salting you won’t be able to log in. To read more see:
<optional> Database stuff:
Ideally this doesn’t happen if you are creating all new tables when going to 1.9.19 and 2.2.7 but if you get any errors:
- if you get an error "illegal mix of collations" this is an issue where there was a change to some tables from utf8_unicode_ci to utf8_general_ci
- ran alter statements on every table listed in the join statement that is shown in the error message. a statement like this:
"alter table 'mdl_wiki_pages_old' convert to character set utf8 collate utf8_general_ci;"
- if moodle upgrade is successful, when logging into the site remember to use your old admin password from the other site since that is what you just restored.
- Once logged in, click on “Site Administration” and complete the upgrade
- <optional based on moodle version> verify book plugin is installed properly by logging into Moodle and going to "site administration > notifications" to finish install
- <optional> if using paypal, set it up for the courses. Go into "users > enrollment methods > Paypal" and make sure your settings are correct.
- <optional> I had some videos in my classes that I stored in a separate directory in my file system not in the moodle directory which I had to move to the new server. Originally the videos did not display. The way I figured it out was to switch over to the HTML mode in the editor so I could see the code and noticed the path for the video and realized I had not moved that directory over. Once I did the videos played.
- <optional> run the “Assignment upgrade helper” found in the “Site Administration” at the bottom of the list. This will update any assignments you may have.
- <optional> if you have quizzes, then in the run the upgrade. In “Site Administration” click on “Question Engine upgrade helper” and click on “List quizzes still to upgrade” and then click the “convert attempts” link and click the Yes button to convert it. Then click the link “List quizzes still to upgrade” to get back to the list and convert the rest. Once done click on “List already upgraded quizzes that can be reset” and reset the quizzes if you desire.
Run the Maintenance script:
- Run the cron maintenance script by clicking “Site Administration” and then click on “Notifications”. In this window you will see a link for “cron.php maintenance script” click it and let it run.
- While logged into your site as an admin, open a new browser tab and enter in this url:
<IP Address to your site>/admin/tool/health/index.php
This will run a health script. Ideally you get the message “There is no health problem found!” which is what I get. If not address the issues.
Final Important Step after each upgrade:
Although this can be super tedious, I highly recommend you now go through your courses page by page to make sure they are working correctly. I originally found issues with images not appearing, videos not appearing and alignment being center justified when it was supposed to be left justified. I would not have known about if I hadn’t gone through the courses. If there is an issue, you want to fix it now before you move on to the next upgrade otherwise you could make a simple problem more difficult later on.
In the beginning my process wasn’t the steps I have listed above, as there wasn’t a good reference and I ended up doing a bunch of processes mixed together versus a completely “clean” install which includes moodle directory, moodledata directory copy and importing a MySQL dump into an empty database. When I did these things everything went smooth and clean; if I didn’t I got all sorts of messed up results. After doing it cleanly I probably found 5 minor things that needed to be done at each upgrade and 1 important directory move of my own that I had forgot about one the first upgrade (that was the one I had my videos stored in - I forgot to copy that directory to the new server; I caught this by going through the classes and seeing the videos were not loading). Worse case if a problem is found, you can just fall back to Moodle 2.2.7 and do a full refresh based on the steps done above because you have a backup of all the code and data if you were following my steps.
Overall I found maybe 5 alignment issues after each upgrade in total across my six classes. The final upgrade to 2.4.1 actually affected one of my classes with fonts that I had to adjust and one class had an issue with the quizzes in it.
- Fonts: Not sure why but randomly fonts were changed from Veranda 12 point to Arial 14pt. So take the time to go through your classes. Changing the font is a little time consuming but overall a minor issue to deal with after the long upgrade process.
- Quizzes: Not sure why this happened as another class that had quizzes was fine. I resorted to deleting the quizzes and recreating them. Like the fonts, it was more of a minor issue to deal with in order to get to the latest moodle version.
Upgrading to next Moodle version:
I repeated this process going to 2.2.7 which went very smoothly.
Do this step when upgrading to 2.2.7:
Now back in phpMyAdmin, drop all tables in the database, import the edited .sql dump into the database and you should now be up on the latest database engine.
After upgrading to 2.2.7 then do this set of steps (don’t do this until you are at 2.2.7)(Note that I didn’t realize this and did this step after getting to 2.4.1 and it was successful):
Once you have upgraded to 2.2.7, then you should upgrade your database from MyISAM to Innodb.
- Using phpMyAdmin perform a new MySQL dump and save it off with a name like “localhost227_MyISAM.sql” (this is important so that you have a backup).
- Then download this sql file to your local computer so you can use Notepad++ to make some changes.
- Once it is downloaded I renamed it to something like “localhost227_Innodb.sql”.
- Now open it up and do a “Search and Replace” where you search for 'ENGINE=MyISAM' and replace it with 'ENGINE=Innodb'. I also used the option “Match whole word only” in order to make sure nothing gets changed except for this engine line statement. This file will be big so there’s always a chance there could be something else so I isolate the change to this. It should basically make a change for every create table statement so there will be a lot.
- Save the file when done.
- Upload the file back to your server.
- Using phpMyAdmin drop all your tables (follow the steps earlier in this document if you need help here). Make sure all tables get dropped; I had two pages worth of tables and ended up only dropping the first page and was confused why it didn’t drop all the tables and that is when I realized there were more tables I missed, so I dropped these ones too. The entire database should be empty now.
- Now import the “localhost227_Innodb.sql” file. Your database should be upgraded to Innodb now.
Upgrading to 2.3.4 and 2.4.1:
I was told the jump to 2.3.4 and 2.4.1 was less difficult and only the code base needed to change. So the process was a lot easier but backups are highly recommended in case you need to fallback due to unforeseen errors (I had none).
**Note that this last approach for 2.3.4 and 2.4.1 is up to you. You may want to follow the same process above like I did for all the other courses. One interesting twist I ran into was that one of my course’s quizzes didn’t work in 2.3.4 that worked in 2.2.7. The only thing different was the upgrade approach so keep that in mind. Otherwise the rest of the course looked fine for pages, content, alignment, images and videos. Not sure what the deal was with the quizzes in that one course (quizzes in another course were fine). So I decided to just delete the quizzes in the one course and recreate them when I was completely done with the upgrades. Minor task to deal with in order to get successfully to the latest and greatest version of Moodle.
The steps I did for this upgrade were as follows:
- Performed a MySQL dump of the database for a backup and named it accordingly (putting on the version name at the end of the file so it looked like: localhost2_2_7.sql)
- Copied moodledata to another directory and renamed the copy to “moodledata2_2_7”
- Uploaded moodle 2.3.4 onto the server as “moodle2_3_4”
- To switch/upgrade versions I now renamed my current moodle directory (which was 2.2.7) to “moodle2_2_7” and renamed “moodle2_3_4” to “moodle” which now put the 2.3.4 code base as the main moodle version now.
- Now I ran the install: <IP address>/moodle/install.php
- This ran through the install and everything was successful. At the conclusion I was presented with the moodle login window. Before logging in you have to update the config.php file with the previous password salting.
- Opened the old config.php file from the “moodle2_2_7” directory and copied the passwordsaltalt1 line and the passwordsaltmain lines to a text file (like notepad) and renamed passwordsaltmain to passwordsaltalt2.
- Opened the new config.php file in the moodle directory and before the passwordsaltmain line I added the two passwordsaltalt lines so now I have three password salt lines in the current config.php file:
- Save and close the config.php file and now you can log in.
- Go through your courses making sure they look fine.
- Repeat this shortened upgrade process to go to Moodle 2.4.1
- Finally you are done!
Run maintenance script:
Once you are doing your upgrade and verified your courses are fine. Then I recommend going to “Site Administration > Notifications” and clicking on the link to run the “cron.php maintenance script”. This will do some import cleanup work and other stuff. It may take a little while to run. Once complete it will dump a log to you and it should have executed successfully.
Good luck! Hope this helps!
Thank you so much for sharing this information. I'm in almost the exact same boat, as we are on a hosted provider, currently at 1.9.5 and looking at relocation at least to a different host at the same provider to get supported versions of MySQL and PHP to make the jumps to current 2.x code. This may save me hours of research and trial/error.
Ideally it will save you a ton of time. There are actually a lot of people out there in our same position and the information is scattered and only tidbits here and there. I kept the notes to help myself going thru each upgrade since I realized it was going to be repeative and then wanted to share and help others since some helped me (like Ken Task who was an incredible help). The thing that saves a ton of time and headaches is simply accepting the fact that you need to do MySQL dumps and import into new empty databases for the first few upgrades. All the other options have side-effects to your classes. This was the cleanest. Once you do the first couple you realize how easy it is and then the MySQL dump approach becomes a great backup mechanism. There is also the tidbit information that is not well documented where this post will ideally help people - like the config.php changes, the password salting and otheres. The only thing I haven't done yet is switch the DNS as I'm still making some final updates since I completed the upgrade. That will require a MySQL dump, edit the sql file to change the current <IP Address> to be my domain name and import it again. Ideally this final step goes smoothly. ;)
Good luck and I hope these notes help you out!
Shawn, thanks for providing the great information about how you upgraded your Moodle.
It appears that you have succeeded without the need of doing anything from a SSH session. Is my observation correct?
It appears that you managed all of these upgrades by using FTP, and the utilities provided by your hosting company.
If so, what are your thoughts about doing some of this from a SSH session?
I am just trying to pick your brain a little more, hope you do not mind.
In a shared hosting environment those options aren't available, that's why there are a lot of challenges for people in hosted environments and what I had to figure out. But as you can see from my documentation it is all possible using the few tools I listed.
If you are in your own environment where you have command line access to your linux servers, then sure use Putty log in and move things around. It's probably easier in many ways (speed wise) and you can also use git more effectively if you want to take that route. Otherwise what I posted will definitely work for anyone I think.
I had a shared hosting account around 3 years ago, and I too was able to upgrade using FTP and the hosting utilities. Now that I have a VPS, using SSH becomes much easier. But it is good to know that it can be done either way.
With both my hosted accounts (I have 1 VPS, and 2 hosted accounts with different companies), both now provide SSH. Maybe little by little, as the competition heats up, more companies will provide SSH with their hosting accounts.
Thank you, I will read it thoroughly.
Good share, the possible flaw with this is handling non default plugins and modules as this approach may not cover that completely.
what was your experience in this regard?
I did have two plugins with the original course in version 1.9.5, one was the "book" plugin and a "feedback" module. I do mention this briefly in the "Starting steps" section. The book plugin has to be installed during each upgrade until 2.3 as then it is included in moodle. Download the plugin for each version of moodle. After uploading the module directory structure to your server, then move the plugin to the appropriate directory (per the plugin install instructions) and then after the upgrade of moodle is done and you login, go to the Site administration and see if there are any configuration settings that have to be addressed. For the book plugin I had to do this for each version but once you get to 2.3 you don't have to deal with it anymore. The feedback module I was using wasn't maintained after 2.0 and so I just dropped that and recreated this one with the new feedback module in 2.4.1
If you are this far back like I was there are going to be some issues where a plugin or module may have limited upgrades. By the time you get to 2.4.1 you will find a much better presentation with a lot more plugin and module options built into moodle by default which is really nice.
Thanks for the detailed explanation. We installed with SimpleScripts, and it does not appear we upgrade automatically. Our current version is 2.3.1.
Are there security issues that make it necessary to continually upgrade?
Yes there are security improvements along with a variety of feature updates. Like any software product, updates are always coming out and you really have to stay current; if not, eventually you'll find yourself in a situation where you are too far behind. I'm in software development myself and let myself get behind with moodle and it become a huge project to get current. It's best to try and stay on the latest version if you can.
I really appreciate your notes. I'm in the exact same situation except I'm starting with 1.9.4. I have several questions but will post them as I come to them in the upgrading process.
First of all, let me say that I am not savy with server software and the such, which may be why I have some questions.
I'm in the starting steps. Correct me, but it seems that in the process I download my moodledata file (which I believe in my case is named onlineclasses) rename it and upload it to my root directory. Simply put, it appears I am copying the file from public_html to the root directory and renaming it for the sake of having a backup copy. Am I correct?
Thanks for your help and patience as I work through this.
One other significant difference is that I am not switching web hosts. So what I have done is created a second moodledata file named in accordance with the upgrade (i.e. Moodle1_9_19) while keeping my original (onlineclasses) as is.
Can I do this and follow the "New Server" information with my new folder and the "old server" information for my original folder? Would it work for me to do all my upgrades this way and then change the final upgrade folder (Moodle2_4_1) to "onlineclasses." I will have a backup copy of the original "onlineclasses" locally on my computer and in my root directory.
I still have a few students working and I do not know how long it will take me to do this especially since I am new at upgrading (and nervous). My thought was to keep the original ("onlineclasses") untouched until the end so as to cause as little down time as possible.
I'm by no means and expert at this. I just documented what I did in my specific situation - and that included moving servers. So I can't guarantee that my steps will completely work for you, however there is no reason you shouldn't be able to have two installations of moodle. You would need to make sure your moodledata is named different, your database is named different and your config files reflect the correct paths. If you are on the old 1.9 versions like I was, a major consideration to be aware of before you ever start this process is to make sure you can upgrade your server environment to the specs required for 2.4, otherwise this long process will be a waste and you won't get very far - that's why I had to switch hosting providers and the reason for the "server switch".
Thanks. I did double check the requirements and we are fine in that regard. Before I go much further I will reread your steps and try to tweak them to my situation. Hopefully, if I can ascertain what you did and why, then I can have a grasp of what I need to do.
I think my biggest hurdle is uncertainty and the fear of loosing everything. Please check back as I may have some questions. I plan on keeping a journal and posting something like you did for this specific situation. It took many hours/days to find the help you have given here.
I would expect the process to work for you, the only thing I can think that you may have an issue with is a potential conflict with the config.php file as that points to where all the directories, files and database are. The key is keeping everything separate and unique. A huge factor in the success I found was backing up and restoring the courses using MySQL dumps. And keep backup copies of the dumps and moodledata at each level as that will enable you to go backwards if you have any issues. Until I figured out the process, I had to revert back a couple of times.
Thanks for all the info. It was a great help. I have been able to upgrade from 1.9.4 to 1.9.19 to 2.2.9+ to 2.3.7+ to 2.5+ The info you gave was great. Since my situation was a little different obviously somethings in the process were different. I plan on posting them like you did including how I changed the default latin1 charset in sql to utf-8 for the 2.5+ upgrade.
I only have one problem left - my email still doesn't work. I posted for help in the general forum - https://moodle.org/mod/forum/discuss.php?d=229901