Hello:
I am trying to upgrade to 1.8, but I get the error message below. I converted to the UTF-8 database when I first installed 1.7. I have upgraded to the latest version of 1.7 (twice), but I cannot find any place to perform the database upgrade again.
I have tried to do the upgrade myself on my mac using the
iconv -f original_charset -t utf-8 moodle > moodlenew
command, but I get this error:
iconv: conversion from original_charset unsupported
Any suggestions?
Jay
In reply to Jay Melton
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Stephen Wilkinson -
I can't help - I'm having exactly the same problem.
We were running Moodle v1.7+, had already converted to Unicode, buit have now run into problems when updating to v1.8+.
Interestingly, I did not have this problem when uprading another Moodle installation that I manage.
I have checked the database to make sure that all tables and the database itself are of collation utf8_general_ci. They are, and yet I have still have the same error.
Anyd advice would be most appreciated...
In reply to Stephen Wilkinson
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Helen Foster -
Hi Jay and Stephen,
Assuming you've not done so already, please could one of you add a report of your problem in the Tracker, providing as much detail as possible to enable developers to reproduce the problem, then post the bug number in this thread.
Assuming you've not done so already, please could one of you add a report of your problem in the Tracker, providing as much detail as possible to enable developers to reproduce the problem, then post the bug number in this thread.
In reply to Helen Foster
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Jay Melton -
Hi Helen,
Rather than post a new bug report, I did a little digging to see if one was already posted.
MDL-9210 seems to be the same as Stephen's and my issue.
If you would like me to post a new one, let me know, and I will do that. Please let me know what else I can do to solve this.
Rather than post a new bug report, I did a little digging to see if one was already posted.
MDL-9210 seems to be the same as Stephen's and my issue.
If you would like me to post a new one, let me know, and I will do that. Please let me know what else I can do to solve this.
In reply to Stephen Wilkinson
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Jeff Wood -
Not sure this will help, but in the admin folder there is a file called utfdbmigrate.php. Anything happen when you call it up?
Jeff
Jeff
Hi Jeff,
Thanks a bunch for the suggestion. When I called up utfdbmigrate.php, I got:
unknown lang pack detected en_us_utf8
unknown lang pack detected es_utf8
and the error message posted below in the graphic. Are the two unknown language packs causing the problem? I downloaded and installed them in February. I don't have an immediate plan for Spanish, and I can deal with the different spellings.
I have found another problem in all this. In each of the three times I tried to upgrade (and failed) to 1.8, I was locked out of the admin page when I went back to 1.7.2. I could not get the Gordon Bateson technique of deleting my user id info in the database and inserting it again to work. In all three cases I had to reinstall the database I thankfully backed up.
I would suggest holding off on upgrading until these problems can be worked out.
Jay
Thanks a bunch for the suggestion. When I called up utfdbmigrate.php, I got:
unknown lang pack detected en_us_utf8
unknown lang pack detected es_utf8
and the error message posted below in the graphic. Are the two unknown language packs causing the problem? I downloaded and installed them in February. I don't have an immediate plan for Spanish, and I can deal with the different spellings.
I have found another problem in all this. In each of the three times I tried to upgrade (and failed) to 1.8, I was locked out of the admin page when I went back to 1.7.2. I could not get the Gordon Bateson technique of deleting my user id info in the database and inserting it again to work. In all three cases I had to reinstall the database I thankfully backed up.
I would suggest holding off on upgrading until these problems can be worked out.
Jay
Hi Jay,
I could only think of this possibility... You have performed unicode migration, after which moodle sets unicodedb to 1 in your mdl_config table. You then have dumped the migrated db to a newly created db using 'create database abc' without specifying 'character set utf8'. Thus the new db has the encoding in latin1, all table encodings in unicode. When you try to perform the migration again, Moodle checks the mdl_config table and finds a migrated flag, preventing another migration.
Could you please check your table and db encodings and confirm is this is the case?
Cheers,
Yu
I could only think of this possibility... You have performed unicode migration, after which moodle sets unicodedb to 1 in your mdl_config table. You then have dumped the migrated db to a newly created db using 'create database abc' without specifying 'character set utf8'. Thus the new db has the encoding in latin1, all table encodings in unicode. When you try to perform the migration again, Moodle checks the mdl_config table and finds a migrated flag, preventing another migration.
Could you please check your table and db encodings and confirm is this is the case?
Cheers,
Yu
Hi Yu,
Thank you for helping with this. I'm not up to speed on all you wrote in your reply/request, but I will try my best to explain.
I installed a fresh version (new mysql db) of 1.7 in January (no students until next week) and upgraded when 1.7.1 came out. My memory is fuzzy on which version, but know I went through the migration process. Yesterday I noticed that 1.8 was out of beta, and tried to upgrade (see the first message in this thread). When that didn't work, I installed 1.7.2. At that point I got locked out of the admin page and worked to figure out why. The fix Gordon helped someone else with did not work for me. I restored the 1.7.1 db using the Backup/Restore function of cpanel. I did this three times. Since the backed up database was already migrated, I wonder if that is an issue.
I cannot see the db right now with phpmyadmin, but I will post again when I can get access.
Before I forget, in checking into the utf8 language modules that popped up on the error page, I see there are TWO directories for languages: home/moodledata/lang AND home/moodle/lang. Which one should have the language packs? What should be in the other directory? I have duplicate copies of three in both, just in case. Is that the way to do it (no languages are showing up in the page now).
Jay
Thank you for helping with this. I'm not up to speed on all you wrote in your reply/request, but I will try my best to explain.
I installed a fresh version (new mysql db) of 1.7 in January (no students until next week) and upgraded when 1.7.1 came out. My memory is fuzzy on which version, but know I went through the migration process. Yesterday I noticed that 1.8 was out of beta, and tried to upgrade (see the first message in this thread). When that didn't work, I installed 1.7.2. At that point I got locked out of the admin page and worked to figure out why. The fix Gordon helped someone else with did not work for me. I restored the 1.7.1 db using the Backup/Restore function of cpanel. I did this three times. Since the backed up database was already migrated, I wonder if that is an issue.
I cannot see the db right now with phpmyadmin, but I will post again when I can get access.
Before I forget, in checking into the utf8 language modules that popped up on the error page, I see there are TWO directories for languages: home/moodledata/lang AND home/moodle/lang. Which one should have the language packs? What should be in the other directory? I have duplicate copies of three in both, just in case. Is that the way to do it (no languages are showing up in the page now).
Jay
In reply to Jay Melton
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Stephen Wilkinson -
Yu Zhang has helped by suggesting the config table in the database.
I have successfully upgraded one Moodle instalation and one has not been successful. The successful installation had the unicodedb field set to "true"; the unsuccessful one did not.
I have set the unicodedb field to true and now see the welcome message asking me to confirm if I want to upgrade to v1.8+. Hurrah!
Just going to check the database again and then click YES.
In reply to Stephen Wilkinson
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Stephen Wilkinson -
The unicodedb flag in the mdl_config table was the problem for me - all now upgraded and working.
It needs to be set to "true".
Thanks to all.
In reply to Stephen Wilkinson
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Jay Melton -
Just to be sure on what I need to do:
—backup database
—go to the mdl_config table
—change all instances of latin1_swedish_ci to utf8_general_ci
—change the unicodedb flag in the mdl_config to true
Is that correct? Do I just type in the word 'true'?
—backup database
—go to the mdl_config table
—change all instances of latin1_swedish_ci to utf8_general_ci
—change the unicodedb flag in the mdl_config to true
Is that correct? Do I just type in the word 'true'?
In reply to Jay Melton
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Stephen Wilkinson -
You need to
1. Backup database
2. change all instances of any encoding (for fields, tables and the entire database) to utf8_general_ci
3. Go to mdl_config table
4. Change the unicodedb record to read "true" in the value field (no quotes) - just type "true".
5. Then try again with the upgrade.
Best of luck.
In reply to Stephen Wilkinson
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Jay Melton -
Thank you all for working with me on this. I'm afraid my skills with the db are not at this level. When I went to the db with phpmyadmin, every instance of the collation is set to latin1_swedish_ci (that's 163 tables!) I went to Operations and hit the Go button for changing the collation to utf8_general_ci. That move changes ONLY the sum value at the bottom to utf8_general_ci. All the rest of the tables remain latin1_swedish_ci.
When I try a search for the string latin1_swedish_ci, I get 0 hits. Sorry to ask this, but someone give me specific instructions on using phpmyadmin to change the character settings?
When I try a search for the string latin1_swedish_ci, I get 0 hits. Sorry to ask this, but someone give me specific instructions on using phpmyadmin to change the character settings?
In reply to Jay Melton
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Stephen Wilkinson -
It does not sound as if you have previously used Moodle to change to unicode.
I would go back to your Moodle v1.7.2 installation and use the tool included to perform the unicode migration. This should change the entire database to unicode. (It is worth checking over the database afterwards though.)
To get your previous installation to work you will need to:
1. Make a copy of your config.php file
2. Rename current database (you will probably want to delete this later, but better to be safe and keep it right now)
3. Rename current Moodle v1.8 folder (don't delete as you will need this when you upgrade)
4. Restore old database. Check that the name of the database is correct - cPanel sometimes gives restored databases interesting names. Check also that a database user exists that has admin rights over the database. This user needs to have the same name as the one mentioned in your config.php file.
5. Install v1.7.2 files into your normal Moodle folder
6. Check/edit and then copy config.php into moodle folder
Once installed you will need to run the unicode migration utility and wait... (it can take a long time).
Once complete you will need to:
1. Check the database to see if it is all utf8_unicode_ci. I had to change a few of my tables.
2. Make a backup of the migrated database.
3. Rename your v1.7.2 folder (don't delete it - you may need it again if things don't work out!)
4. Rename your v1.8 folder to your normal Moodle folder.
5. Check/edit and then copy config.php into moodle folder
Best of luck!
In reply to Stephen Wilkinson
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Jay Melton -
Stephen,
Thank you very much for the detailed instructions! I ran the migration script (twice) when I installed 1.7 the first time and either the first or second upgrade. The only thing I can think that I did differently in the 1.7 era is add ja, es, and us_en utf8 languages to my moodledata directory (I wrote about that in another part of this thread) after the initial install. I did this because the install I had could not download the language packs.
I got a bit antsy about getting the system up and running and used the brute force method: I backed up the four courses (no student data until next week) I had created, installed a fresh copy of 1.8 and restored the four courses. All seems to be working well.
Thank you again,
Jay
Thank you very much for the detailed instructions! I ran the migration script (twice) when I installed 1.7 the first time and either the first or second upgrade. The only thing I can think that I did differently in the 1.7 era is add ja, es, and us_en utf8 languages to my moodledata directory (I wrote about that in another part of this thread) after the initial install. I did this because the install I had could not download the language packs.
I got a bit antsy about getting the system up and running and used the brute force method: I backed up the four courses (no student data until next week) I had created, installed a fresh copy of 1.8 and restored the four courses. All seems to be working well.
Thank you again,
Jay
In reply to Stephen Wilkinson
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Tom Worthington -
I have seen numerous references in Moodle documentation to a tool to perform unicode migration. However I can't see such a tool listed in my Moodle 1.7 installation. Where should it be? Where does it appear on the Administration menu? What is it named?
In reply to Yu Zhang
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Stephen Wilkinson -
After I had the problems with upgrading I checked all fields, tables and the database itself - all were encoded utf8_general_ci. And yet it still would not upgrade.
I didn't check the mdl_config table though 
Would a negative value here stop the upgrade process, even if the database was all unicode? How might this value not be set?
Okay, Yu,
I got to the db and the mdl_config table. The collation of the name field is latin1_swedish_ci, so that must be a problem. I cannot find an entry for unicodedb in the mdl_config table.
How can I fix this?
Jay
I got to the db and the mdl_config table. The collation of the name field is latin1_swedish_ci, so that must be a problem. I cannot find an entry for unicodedb in the mdl_config table.
How can I fix this?
Jay
In reply to Jay Melton
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Stephen Wilkinson -
If you're using phpMyAdmin you can change collation of fields by clicking on the pencil (edit) icon. I changed all of mine to utf8_general_ci (this is the collation used when Moodle updates to unicode).
Remember that you also need to change the collation of the table itself (click on operations when the table is selected).
Remember that you also need to change the collation of the database itself (click on operations when the database is selected).
I'm not sure why you do not have a unicodedb record. It is about the 200th record - are you sure you've scrolled through the table enough? You can search for like '%unicode%' in the name field to find it quickly. If you've been working with v1.7+ it should be there!
Hi there
can I just clarify something here.
If all I need to do to change the database on the third party modules/blocks is click the pencil Icon in the php_myadmin on the appropriate tables, e.g. mdl_object, then why does the Moodle docs page http://docs.moodle.org/en/Converting_files_to_UTF-8 say:
can I just clarify something here.
If all I need to do to change the database on the third party modules/blocks is click the pencil Icon in the php_myadmin on the appropriate tables, e.g. mdl_object, then why does the Moodle docs page http://docs.moodle.org/en/Converting_files_to_UTF-8 say:
'For Windows, a free tool doesn't exist. However, the conversion may be done by using [Cygwin], a Linux-like environment for Windows, and excecuting the iconv command in that environment.'
I'm really struggling with these third party bits and the database migration so any thoughts would be useful.
Thanks
Heather
In reply to Jay Melton
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Erick Barreto -
Unicodedb create the registry with a value of 1, add a new id manually.
In reply to Jay Melton
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Stephen Wilkinson -
Interestingly I get the same issue - locked out of v1.7+ when I switch back to the old version. I can log in as the Moodle administrator but do not have administrator rights.
Interestingly, I tried logging in as teacher and students at this point and found that rights were incorrect for all. Is this the roles tabes? The roles tables were one set of tables that were incorrectly collated as latin when I checked the database after v1.8+ could not install.
Is the v1.8+ upgrader changing the collation of tables? Under my MySQL configuration (working with a remote server and cPanel - like Jay) the character set and collation is set to utf8 for "client" but latin for the "global value". Would this cause a problem when creating/updating tables? It didn't cause a problem when I upgraded my other Moodle installation and this is on the same system configuration.
Like Jay, I have had to restore the backup database before v1.7+ would work again for me.
Jay,
I'm glad that you have had some help from those that are much more knowledgeable than I.
I did a new install of moodle 1.7.1 using Fantastico and had a similar issue. I deleted the database that Fantastico created and created another and installed moodle the old fashion way and all was fine.
Jeff
I'm glad that you have had some help from those that are much more knowledgeable than I.
I did a new install of moodle 1.7.1 using Fantastico and had a similar issue. I deleted the database that Fantastico created and created another and installed moodle the old fashion way and all was fine.
Jeff
In reply to Jay Melton
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
Good morning from Manila.
I actually have experienced the same problem. After trying to upgrade moodle 1.7 in my site to 1.9 the data base wont connect to the higher version because of certain incompatibilities. So I went back and installed 1.7 again but I cannot be directed to the front page anymore. I was even locked out of the admin page.
I was planning to the just back up the classes individually and then install a fresh copy of 1.9 then restore the classes. Problem is I cannot access them anymore.
I'm actually hosting at siteground.com.
I actually have experienced the same problem. After trying to upgrade moodle 1.7 in my site to 1.9 the data base wont connect to the higher version because of certain incompatibilities. So I went back and installed 1.7 again but I cannot be directed to the front page anymore. I was even locked out of the admin page.
I was planning to the just back up the classes individually and then install a fresh copy of 1.9 then restore the classes. Problem is I cannot access them anymore.
I'm actually hosting at siteground.com.
In reply to Jay Melton
Re: UTF-8 upgrade need preventing upgrade to 1.8 from 1.7.2
by Robert D. Smith -
http://YOURMOODLESITE/admin/utfdbmigrate.html (Seems like utf8 conversion command center for Moodle 1.72)
http://YOURMOODLESITE/admin/utfdbmigrate.php run this on your 1.72 site and it should upgrade your database to unicode , BUT ALSO BACKUP your database first!
Cheers!
Robert D. Smith
Also check out the other scripts in http://YOURMOODLESITE/backup/db and http://YOURMOODLESITE/blocks/db in your file manager and (backend of web site, make copies and experiment with these scripts!!)