upgrading from 1.9.10 stuck

Re: upgrading from 1.9.10 stuck

by Ken Task -
Number of replies: 11
Picture of Particularly helpful Moodlers

+1 to what Howard said.

Hope you are doing the 'March' on copies and that you are using a fresh higher version code folder with the config.php file of the old site.   Are you attempting to copy over Moodle code directories/files?   If so, those incompatible blocks (the ones for 1.9 only) are still there.    Suggest using only the config.php file from the old site in a completely fresh code directory.

If those blocks exist for the version of 2 you are upgrading to, download an appropriate compatible zip and manually place the zip file in blocks and unzip.   You are then kinda upgrading the site and at the same time upgrading those add-on items.

One other way without some serious/guruish DB work, while the site is at version 1.9.19, remove those blocks.  Then move forward one.

Do full site backups at each successful step upwards in the 'March'.   Those become fall back points should the next step hickup.

'spirit of sharing', Ken

In reply to Ken Task

Re: upgrading from 1.9.10 stuck

by john owen-jones -

I was able to reach 1.9.19

my current procedure is to ftp the moodle install to the server and put config.php in to the install folder

which starts the upgrade process.

currently I have several moodle folders

moodle (currently 1.9.19

moodle1.9.10

moodle 2.0.10

moodle2.2.11

I initially tried to go from 1.9.10 to 2.0.10 but got the complaint about the nonstandard blocks admin and admintree (which are not part of 2.0.10) and it said not on disk

so i then tried 1.9.10 to 1.9.19 which was successful.

I found a post which seemed to suggest 1.9.19 to 2.2.04 worked and as I could only find 2.2.11 I tried that but it just gives a blank page. 

So I currently have a working 1.9.19 but no apparent way to upgrade it.  I think there may be an issue with the database as older tables are in myIasm format and newer moodle Requires innoDB

If I cant upgrade I may choose to create a new install of 2.0.10 and see if i can transfer the old db contents to the new database. Other wise I would have to manually recreate the existing moodle site last time it took 6 weeks I'm hoping someone has a way of overcoming this issue.

I've just seen a post where any references to admin and admintree were removed from the database and It looks like that was successful. I think

 

In reply to john owen-jones

Re: upgrading from 1.9.10 stuck

by john owen-jones -

Well I upgraded all the tables to innoDB and removed references to admin and admin_tree in block table and dropped the mdl_upgradelog table.

now all my plugins are fine but I get

System

Debug info: INDEX command denied to user 'moodleuser'@'X.x.x.x' for table 'mdl2_upgrade_log'
CREATE INDEX mdl2_upgrlog_tim_ix ON mdl2_upgrade_log (timemodified)
Stack trace:
  • line 397 of /lib/dml/moodle_database.php: ddl_change_structure_exception thrown
  • line 669 of /lib/dml/mysqli_native_moodle_database.php: call to moodle_database->query_end()
  • line 88 of /lib/ddl/database_manager.php: call to mysqli_native_moodle_database->change_database_structure()
  • line 75 of /lib/ddl/database_manager.php: call to database_manager->execute_sql()
  • line 475 of /lib/ddl/database_manager.php: call to database_manager->execute_sql_arr()
  • line 93 of /lib/db/upgrade.php: call to database_manager->create_table()
  • line 1393 of /lib/upgradelib.php: call to xmldb_main_upgrade()
  • line 273 of /admin/index.php: call to upgrade_core()
 username and Ip address omitted / changed
 
In reply to john owen-jones

Re: upgrading from 1.9.10 stuck

by john owen-jones -

I have a feeling that my db user doesn't have enough rights in the db on our hosting so I think if I copy the site to a local debian box where i can have all the rights i need upgrade the db then move it back on to the host maybe.

In reply to john owen-jones

Re: upgrading from 1.9.10 stuck

by Ken Task -
Picture of Particularly helpful Moodlers

Even if you get the migration to work on local debian box and then restore that work onto the hosting environment, because the DB user for Moodle doesn't have access rights enough, you'll be back here reporting some stuff that can't be fixed in Moodle code.    Best, me thinks, working on getting the DB user on your production server with the access rights it needs.   Contact hosting provider.

Please see:

http://docs.moodle.org/20/en/Create_Moodle_site_database#MySQL

and on that page:

mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO moodleuser@localhost IDENTIFIED BY 'yourpassword';

'spirit of sharing', Ken

 

In reply to Ken Task

Re: upgrading from 1.9.10 stuck

by john owen-jones -

hi ken,

thanks for the information it is going to be useful.

i'm upgrading two colleges they are sisters but generally do things there own way initially I was brought on board to sort out the first colleges moodle. What had happened was they had one site one year and then started a new site the following year unfortunately things went wrong files were getting lost and it was a disaster. It turns out they had used the same database and tables for both versions.

so what I did for the first site is install 2.61 on a debian server and replicate the original site by hand and course by course downloading the course backup extracting the file resources and creating new url's and pages.

this was good for them and for me as it gave me a chance to become familiar with moodle and its workings I've been a linux geek for years and used a few cms systems but I had zero moodle experience.

After my test server was built I migrated it too another debian box this box i told the apache server the site name was the same as the public site and used hosts on both that system and a client on my lan to fool them into thinking they were on the internet most of the site was ok but for a few hard links to the original.

I backed up that site and uploaded it replacing the original moodledata and moodle directories on the public server and then used mysqladmin to create the tables on the server. A bit of a pain to do as I couldnt do a straight restore. But used geany to load the mysql backup file alter the table prefixes and got copying and pasting until i had gone through the file (that was about 13.5 meg in size) and then changed config.php to use the new table prefix.  There was a bit more work involved in making the site mobile friendly configuring it to work with gdrive and adding back all the current teachers and students but its up and working, fine for now.

I don't really expect it to remain fine once 600 students are trying to use it, I think the hosting is not up to the task however we have a big change coming in the college broadband which will make it practical to host the servers ourselves.

The second college has exactly the same hosting plan as the first but never got messed up in the same way apart from 13,500 users/spambots  writing blog posts from around the world since deleted and i've added recapture to at least slow them down at least.  This site should be suitable for the 'march' however as you can see i've come a bit unstuck as i've been unable to upgrade beyond 1.9.19  yesterday I backed up the mysql database and did a find and replace to change the table type and gave the tables a new prefix and then used phpmyadmin to recreate the data in new tables. That was around 80M of database backup but as I didn't need to restore the massive log file or the thousands of blog posts I managed that in 3hours of copy paste. I moved the moodle site on to the new database tables and its working fine.

since 2.6.1 is working on the hosting It should be the same for this second site if I can complete the march successfully off the public host and then proceed as for the first site.

As the current hosting has worked for the last couple of years, it should work at least until I can get better servers sorted out for the two sites.

I'm just hoping I can do the march using my test server and proceed as planned otherwise I can see another 6 weeks of site rebuilding again.

 

 

Average of ratings: Useful (1)
In reply to john owen-jones

Re: upgrading from 1.9.10 stuck

by Ken Task -
Picture of Particularly helpful Moodlers

You've already kinda said the current hosting package may not handle it with traffic.   But same could be true without traffic on some scripts - example: DB for sessions as opposed to file - some course backups.

When assisting many with upgrading from 1.9.x to 2.x, one of the items found to be needed was some tweaking to MySQL config.    In checking logs, found that max_allowed_packet defaults not enough to migrate the site.   Once that was increased, no issues.  Since 2.x requires innodb, settings for those are also in need of tweaking: innodb_buffer_pool_size=?G

MySQL, when first installed, offers cnf files for small, medium, large sites ... the default is small cnf.  Check out the other cnf files (I use CentOS not debian).

Suggest you acquire mysqltuner and run it on the debian development setup.   Do that on a DB that you hadn't cleaned up.   Note the recommendations/warnings.   Then do same run on a DB that has been 'cleaned up'.

Of course, that's not on a site that has true traffic but ...  One should run tuner ... make changes ... then wait a week or so to let MySQL work with the new settings ... then run it again to see how it's performing.

Also, to 'future proof' upgrades/updates to code in future, strong suggest using git to install the initial whatever.  Hopefully, your remotely hosted package allows shell access (but it does sound like those might be shared packages).  May not be able to take advantage of git on the hosting site, but should be able to do so on the local debian dev box.

2 does more requires more ... period.   On 1.9 one could get away without any PHP optimizing (depending upon site size/traffic).  Version 2 by the sheer fact it now includes MUC itself, indicates if you want better performance either APC or ZendOpCache is needed.   Would go with ZendOpCache as that will be included in versions of PHP greater than 5.3 now.   That's another item to explore ... and can't really be done on a dev box ... has to have real traffic.

By all means check the requirements for the latest and greatest version even IF you will not run it yet.

'spirit of sharing', Ken

Average of ratings: Useful (2)
In reply to Ken Task

Re: upgrading from 1.9.10 stuck

by john owen-jones -

Hi Ken,

I'm a fairly happy bunny today I have managed to complete the march to 2.6.1 from 1.9.10. on my test server at least. Your help and advice has been very encouraging and helpful. 

In the spirit of sharing this is how i did it version 1.9.10 > 1.9.19 > 2.0.10 > 2.2.11 > 2.3.11 > 2.5.4latest to 2.6.1latest. My test server will take 2.7 but my host is limited to 2.61 so until hosting is changed this is my limit.

I think the big lesson to be learned from this is don't do this on your live server! you will go from e.g 1.9.10 to 2.61 as in my case but you will be restoring the site from a backup of your testserver which starts with a backup of your live site gets upgraded and then this is backed up and transferred to your public site.

Back up the existing site download moodledata and moodle and the current database and recreate the site on a test server with proper access.

I used apache and used 'sites available' to tell it it was serving the live site.

why, because some code, pictures mainly  will be linking to files on the live site address.

I also used hosts to redirect my browser on my client computer to fetch data from my test server

e.g 192.168.1.15 mymoodlesite.com in hosts it won't go looking for it on the internet and moodle won't complain about the wrong site name.

I also had ssh access from my client to my server i and used filezilla for transferring files both ftp from my live site and via ssh to my server.  one thing to note with filezilla in its default configuration it treats files without extensions as ascii as moodle stores files with a string of random looking letters and digits as a file name in later versions with no extension you need to change the default to binary. Otherwise you will corrupt all your docs, powerpoints pdfs and pictures.

 

it also pays to have command line access with mysql

very useful is

mysqldump -u root -p moodle >  2.6.1db.sql

I call each dump by the version of moodle it works with

mysql - u root -p moodle < 2.6.1db.sql

is also very useful you use this on an empty database usually (I did use a dump I called patch.sql which just had the 3 block tables with slightly changed contents it deleted the tables recreated them and filled them).

There is documentation on line for converting to unicode and innodb

mysql -u root -p

gets me into mysql.

DROP DATABASE moodle;

easiest way to clean up replace the moodle by your database name.

CREATE DATABASE moodle;

creates a new empty database  you populate it with your database backup as shown earlier.

SHOW DATABASE;

fairly obvious shows the databases you have created on the system.

USE moodle;

select which database to work with in this case moodle

SHOW TABLES;

gives you a list of tables on the current database.

DROP TABLE mdl_upgrade_log;

this table is a pain if something went wrong even after you fixed the problem 2.0.11 will not proceed to upgrade if this table exists,

GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,CREATE TEMPORARY TABLES,DROP,INDEX,ALTER ON moodle.* TO moodleuser@localhost IDENTIFIED BY 'apassword';

you should only need to do this once its the user and password used in config.php and is essential to letting moodle do anything. change moodleuser and apassword to your requirements.

SHOW GRANTS For 'moodleuser';

should show you permissions for your moodleuser its not very readable but should be ok.

EXIT;

takes you out of mysql and back to your regular command line.

Don't be afraid to do an install instead of an upgrade if you need a copy of a clean set of tables set to defaults

I needed this because my admin block's were missing I looked at the three block tables and found some missing entries I copy pasted these into my copy of the live database. each line is indexed e.g (1, xxx), (2,xx),

 (4,xx) so i know (3,xx) is missing so I add that entry and save the sqldump under a new name and drop the database create it again then fill it again with my modified dump at the command line. This got me my missing blocks back once happy with the site in its current version i back up the database again with its current moodle version number and move on to the next upgrade.

if the upgrade fails I can rename moodle to moodleversion and rename the old version back to moodle restore the database for the old version. check its ok and then try to upgrade again (e.g 2.2.11 was too big a leap from to go to 2.6.1 so i added some intermediate steps 2.3.11 and 2.54latest which did work, (problem was I had lost my password in the jump from 2.2 to 2.6 also unit tests was missing).

So finally I am at version 2.6.1 but it's not finished the old courses are now in legacy format so now its a case of working my way through them adding the missing files which are in moodle data. I can use the live site as a guide so i can see what needs to go where.  Once I've finished that I can upload the new site to the host.

unfortunately the host doesn't give me a command line so i'll have to use phpmyadmin and copy paste my updated database using sql queries i'm writing to clean new tables as if i tried to overwrite the old ones there would be duplicate keys. best practice would be to use a fresh database however you can get by using a new prefix e.g if your old moodle used mdl_ as the prefix then use say mdl2_ as a prefix and when your ready switch to mdl2_ as the prefix in config.php you will need a few changes to config.php as your site root and moodledata root will be different to the paths on your host. But to your website everything looks the same.

Anyway hopefully this post will be of some use to future upgraders. Do not do the march on your live site!it is tedious slow and you may not even have the tools to hand to do the job, once your new site version is ready then you can ftp the new version and set the config.php to look at the right database tables. If it goes wrong you can easily revert to your old site and figure out what went wrong.

hope this post is useful and not too confusing to follow.

I know I skipped  converting to utf and innodb but this should help

http://docs.moodle.org/26/en/Converting_your_MySQL_database_to_UTF8

(i used vi not vim but it works the same)

regards

john

 

Average of ratings: Useful (1)
In reply to john owen-jones

Re: upgrading from 1.9.10 stuck

by Ken Task -
Picture of Particularly helpful Moodlers

Congrats!  And thanks for 'sharing back'.   Your description will be useful to others and I've marked your post useful! ;)

'spirit of sharing', Ken

In reply to Ken Task

Re: upgrading from 1.9.10 stuck

by john owen-jones -

hi ken  you'll like this post

https://moodle.org/mod/forum/discuss.php?d=254419#p1103696

basically the issue was courses migrated with legacy files.

my courses were missing the resources files.

At first I tried downloading them as backup files from my live server and then unzipping and uploading.

after an examination of moodledata I found a interesting quirk to the structure.

basically the numbered folders hold the course files for the corresponding courses.

e..g /moodle/course/view.php?id=230 is courseid 230 and moodledata/230/ holds the files used in that course

change the id of the course and you can naturally work through the courses smile

uploading whats needed for each course simple isn't it

regards

john

 

 

In reply to john owen-jones

Re: upgrading from 1.9.10 stuck

by john owen-jones -

It really shouldn't have been necessary to upgrade course files manually.

somehow my moodledata folder was incomplete, I downloaded it again and just replaced missing or different sized files and repeated the upgrade march this time from 1.9.19 > 2.2.11 > 2.5.4 > 2.61 this time it seems to be nearly fully successful.  The omly issue was there were a few instances of an extended character on some site pages which looks like a capital A with a tilde~ above it.

This has come from the database dump.

 

regards

john

 

In reply to john owen-jones

Re: upgrading from 1.9.10 stuck

by David Bezemer -

Hi John,

the advised version order is:
1.9.latest > 2.2.latest > 2.6.latest

The in between versions of Moodle 2.0.x will cause problems with the current upgrade scripting, as it skips some steps that were added in Moodle 2.2.
For more detailed instruction see: http://docs.moodle.org/26/en/Upgrading which clearly states: "Note: You can only upgrade to Moodle 2.6 from Moodle 2.2 or later. If upgrading from earlier versions, you must upgrade to 2.2 as a first step."

Average of ratings: Useful (1)