Worried! Unicode 1.8.2 to 1.8.4 problem

Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
Number of replies: 22

I started the installation and reached this page:-

Upgrading Moodle database from version 1.8.2 (2007021520) to 1.8.4 (2007021540)...

 
 

Your Moodle files have been changed, and you are about to automatically upgrade your server to this version:

1.8.4 (2007021540)

Once you do this you can not go back again.

Please note that this process can take a long time.

Are you sure you want to upgrade this server to this version?Upgrading Moodle database from version 1.8.2 (2007021520) to 1.8.4 (2007021540)...

 
 

Your Moodle files have been changed, and you are about to automatically upgrade your server to this version:

1.8.4 (2007021540)

Once you do this you can not go back again.

Please note that this process can take a long time.

Are you sure you want to upgrade this server to this version?

I said yes but had warnings as follows:

Name Information Report Status
unicode docs.gifis required to be installed/enabled.

It is required that you store all your data in Unicode format (UTF-8). New installations must be performed into databases that have their default character set as Unicode. If you are upgrading, you should perform the UTF-8 migration process (see the Admin page).

 Check
php_extension mbstring docs.gifis recommended to be installed/enabled

Installing the optional MBSTRING library is highly recommended in order to improve site performance, particularly if your site is supporting non-latin languages.

 Check
php_extension openssl docs.gifis recommended to be installed/enabled

Installing the optional OpenSSL library is highly recommended -- it enables Moodle Networking functionality.

 Check
moodle docs.gifversion 1.0 is required and you are running 1.8.2 OK
database mysql docs.gifversion 4.1.16 is required and you are running 5.0.45 OK
php docs.gifversion 4.3.0 is required and you are running 5.1.6 OK
php_extension iconv docs.gifis recommended to be installed/enabled OK
php_extension curl docs.gifis recommended to be installed/enabled OK
php_extension tokenizer docs.gifis recommended to be installed/enabled

I followed the link in the first warning about unicode and got no simple explanation about what I actually had to do and having followed links a few levels down I still have no idea and my site is now locked in an unuseable state that I can't back out if and it is needed tomorrow. This is a rediculous state to leave a system/user in - I've trawled for answers as many people (different versions) seen to have had the same probelm - I can't see a clearcut answer yet - looks like it will be a night with no sleep.

Can somone please help - I dont even know (or care) what unicode is - I am a 1 man user - I just want to click on upgrade and that's it.

Average of ratings: -
In reply to Neil Turner

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
My webhost has an application inthe control panel PHPMyAdmin and when I open that, among the information it says
"

localhost

  • Server version: 5.0.45-community
  • Protocol version: 10
  • Server: Localhost via UNIX socket
  • User: XXXXX@localhost
  • MySQL charset: UTF-8 Unicode (utf8)
  • MySQL connection collation: utf8_unicode_ci
"
Does this mean my databases are correctly encoded in UTF-8 and the warning I am getting is wrong?
Average of ratings: -
In reply to Neil Turner

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
The warning says:
"If you are upgrading, you should perform the UTF-8 migration process (see the Admin page)."

I am upgrading but I am now unable to go to the admin page but do I have to do anything - I get the impression it is a false warning about the database and unicode - can someone advise before I procede?
Average of ratings: -
In reply to Neil Turner

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Mauno Korpelainen -

Database itself may be unicode but check the tables (collation and char set)with phpmyadmin - you may have some latin tables (usually from non standard activities)

Average of ratings: -
In reply to Mauno Korpelainen

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
Thanks for the reply. I don't think I have done anything that would be described as non-standard and have no idea where to use PHPMyAdmin to wht you suggest. What do I do if I find what you describe - is it something simple I can change using PHPMyAdmin?


Average of ratings: -
In reply to Neil Turner

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Mauno Korpelainen -
You just said "My webhost has an application in the control panel PHPMyAdmin..." - click the link and you should be able to browse tables of your database. If you find any latin tables they can be converted to collation unicode and char set utf-8 (they all should have been unicode already in moodle 1.8.2 but some web hosts using for example fantastico have changed install scripts so that original environment check of moodle is skipped)
Average of ratings: -
In reply to Mauno Korpelainen

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
The installation is through Fantastico - I didn't realise that the this would or could affect the install - I'll have to find out more about that.

I take these are the tables :-
adodb_logsql
mdl_assignment
mdl_assignment_submissions
mdl_backup_config
mdl_backup_courses
etc...
For each one the "collation" is defined as "latin1_swedish_ci" so I guess this is exactly what is wrong - it's certainly nothing I will have set. Do I now change it using PHPMyAdmin? How? There are checkboxes on the LHS I can check but the "Operations" link at tht top right looks as though I can set the "collation" for any new table I might wish to set up but I'm not sure how I would change the collation for all of the existing tables (loads!) Is this what I should b tryng to do?

thanks
Neil


Average of ratings: -
In reply to Neil Turner

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Mauno Korpelainen -

Take first a full backup (mysql dump file) of your database - export tables with phpmyadmin (select files to backup and save it as a sql file)

If you are not sure what buttons to click send a screenshot of your phpmyadmin or search from forums or phpmyadmin documentation...

Now because your database is collation unicode but tables are latin you could try to convert each table to unicode or drop all tables and import them back to database (collation utf8_general_ci or utf8_unicode_ci and utf8 charset).

The latter is quicker, the former is done by mysql code like (press SQL to write the code and GO)

ALTER TABLE `tablename` CHARACTER SET utf8;
ALTER TABLE `tablename` COLLATE utf8_unicode_ci;
(or ALTER TABLE `tablename` COLLATE utf8_general_ci; )

There may still be some problems because we don't know if migration of data in database is done or not but if you have checked (with notepad or wordpad for example) that your sql dump file is ok you can always drop tables and try again.

Edit: and you could ask your host to enable php function mbstring or tell you how to enable it because it may be usefull too.

Average of ratings: -
In reply to Mauno Korpelainen

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
I think this is too complicated for me to undertake - I really don't understand what's being discussed here. I tried top update my website following a warning e-mail sent out from moodle and now my site is unuseable - I can't beleive that the installation script has left me in a situation that I can not back out of. What would happen if I just procede with the insallation and ignore the warning?

Screenshots of PHPMyAdmin are here
Average of ratings: -
In reply to Mauno Korpelainen

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
All this means nothing to me.
In the SQL window do I type

ALTER TABLE `tablename` CHARACTER SET utf8;
ALTER TABLE `tablename` COLLATE utf8_unicode_ci;

or is

ALTER TABLE `tablename` COLLATE utf8_general_ci;

a single command that replaces both the previous commands or is it an alternative to only the second command? So shoudl I type:-

ALTER TABLE `tablename` CHARACTER SET utf8;
ALTER TABLE `tablename` COLLATE utf8_general_ci;

What is the difference?

Do I have to replace "tablename" with an actual table name repeat this command for each of the 187 tables in turn?
Average of ratings: -
In reply to Neil Turner

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Mauno Korpelainen -

Neil,

I was sleeping when you sent these messages but I try to answer most of your questions.

1) This mess is caused by fantastico scripts that let or at least have let you to install moodle 1.8 skipping the environment/unicode check of normal moodle packages from moodle.org. Installing a moodle 1.8 package from http://download.moodle.org/ works ok if database is collation unicode and character set utf8. I have seen this happen several times before - and you can find several cases from forums here.

2) Your phpmyadmin looks ok and when you need to export tables you should find a small check box under selected tables - save as file - and press button "Go" from bottom right corner. Now you should be able to download the backup file to your PC hard disk and to be able to open/edit it with notepad or WordPad. Leave the Structure and Data options to default values.

3) You can import tables back using Import button or use SQL button to run sql queries.  Some versions of mysql need different command for character set and collation but usually both can be altered at the same time. Collation utf8_general_ci and utf8_unicode_ci are optional (both work - you may select the one you want). Yes - if you use this command you need to replace tablename with the real tablename. It may be possible to change collation and character set of tables with phpmyadmin buttons too (haven't tested that)

4) If your site is not working right now don't worry - it is still possible to get it running again - unless you have no backup and have deleted all data from database...and even in the worst case your host should be able to take all data back from their backups (some days back).

Send me a message or email if it looks too hard to handle. We have several helpful people here in these forums and sometimes some advise given by email may cause a new problem - it has happened to me too. The problem with giving exact solutions is always difficult without seeing the site and environment - every site is different. smile

Attachment Image2.gif
Average of ratings: Useful (1)
In reply to Neil Turner

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by PROFessor VAZZ - Brazil -
Hello,

I have moodle 1.8.2. It's ok.
My DataBase has all tables "utf8_general_ci".

The question is, "Do you recommend to do upgrading to 1.8.4 ?

Thanks.
PROFessor VAZZ
Brazil
professor-vazz@professor-vazz.com
http://www.e-ead.com
Average of ratings: -
In reply to Mauno Korpelainen

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
So in the Export window of PHPMYAdmin I can see that I should:-
"Select all" and check the "SQL" radio button

But what do I do about the Structure and Data options (including INSERT type)?


Average of ratings: -
In reply to Neil Turner

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Richard Enison -
NT,

Also, this could be a case of the Renegade Non-standard MySQL Host syndrome, especially if your host is bluehost or hostmonster. See http://tracker.moodle.org/browse/MDL-11743

This should not happen in 1.8.4, because the corrected query for checking the database should be in the code. But maybe the renegade hosts have pulled a fast one again. Or maybe there's another one. Which host is it, if I may be so bold as to ask?

RLE
Average of ratings: -
In reply to Richard Enison

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Mauno Korpelainen -

Good point Richard,

...and if it a fantastico one-click-package anything is possible ("tailored packages" of cheap host fantasticos are not the same as packages from moodle.org so it is possible that the package upgraded through fantastico actually has old buggy code wink)

Average of ratings: -
In reply to Mauno Korpelainen

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
I asked my webhost to look at this thead and comment on it but they've just said Fantastico is nothing to do with them so they aren't going to help in any way.
Average of ratings: -
In reply to Richard Enison

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
Unfortunately that thread (http://tracker.moodle.org/browse/MDL-11743)
contains far far far too much information for me to even try and understand. The level I need requires someone to tell me which keys to press. Is there a way for me to get my website back up and running?
Average of ratings: -
In reply to Neil Turner

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
Is there a way for me to get my website back up and running?
Average of ratings: -
In reply to Neil Turner

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Neil Turner -
Is this the best place/forum to get help with this or is there somewhere else I can or should be asking?

What has caused this problem? Is it a faulty moodle install script, Fantastico,my webhost, me - I don't understand why I have been left in a situation that I can not back out of - I now have a website that is unuseable.
Average of ratings: -
In reply to Neil Turner

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by John Hobson -
If it's any consolation I have exactly the same problem - Fantastico install of 1.8 and Latin tables.

I exported the tables, and dropped them and then reimported, but there is no "collation" field on the import to select (see attachment).

When I reimport they are still Latin.

Any ideas?

Thanks in advance

John




Attachment moodle.jpg
Average of ratings: -
In reply to John Hobson

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by John Hobson -
Well following something I read elsewhere on the big webby thing

I did a sql backup
dropped the tables
backed up the sql backup
searched and replaced all refs to "latin1_general_ci" to "utf8_unicode_ci " and from "latin1" to "utf8" in SQL backup
Reloaded the tables

This seems to have done the trick

Can anyone see any problems with that?

Thanks

John
Average of ratings: -
In reply to John Hobson

Re: Worried! Unicode 1.8.2 to 1.8.4 problem

by Mauno Korpelainen -

It should be ok - sometimes you may need to check/change also character set of the file (utf8/latin) you are importing (in phpmyadmin)

If default character set of database is already utf8 and default collation is already utf8_unicode_ci imported files usually get the default values - or these can be changed with phpmyadmin or  sql like

ALTER DATABASE nameofdatabase DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_unicode_ci;

It all depends on how the mysql file is done and imported back.

Average of ratings: -