HELP: UNICODE is required to be installed/enabled.

HELP: UNICODE is required to be installed/enabled.

by Manuel de la Torre -
Number of replies: 19
I installed Moodle 1.8.2 using a CPanel10/Fantastico Script and in the Servers Checks (Adminnistration/Server/Environment) I am getting an Unicode check error (below!)

What can I do to fix this error? Because I am in 1.8.2 there is no migration utility!, Can I fix the error manually?

The DB was just created. With phpMyAdmin can see that all tables collation is latin1_swedish_ci. In the mdl_config table the unicodedb is set to 1

This is the actual check error msg:

UNICODE is required to be installed/enabled.

It is required that you store all your data in Unicode format (UTF-8). New installations must be performed into databases that have their default character set as Unicode. If you are upgrading, you should perform the UTF-8 migration process (see the Admin page).

Thanks
Average of ratings: -
In reply to Manuel de la Torre

Re: HELP: UNICODE is required to be installed/enabled.

by Manuel de la Torre -
Replying to my self!!! sorpresa

I just converted the database with:

ALTER DATABASE moodle DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci

And now the check error has gone away risa. But the tables are still latin1_swedish_ci. What command can I use to convert them to utf8?

I was thinking in something like the following but its now working

ALTER TABLE * DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci

 

Thanks !

Manuel!


Average of ratings: -
In reply to Manuel de la Torre

Re: HELP: UNICODE is required to be installed/enabled.

by Richard Enison -

MdlT,

I presume by "now working" you meant "not working". I believe you need to replace the "*" in that command with the name of the table, for each table in the database. Rather than type that command in by hand for the many tables in the Moodle database, depending on the database server you have and the other software you have (like text editors), you might be able to do something like capture the output of a command like SHOW TABLES to a text file and use the search and replace feature of your text editor (I know you can do this with MS WORD, but that's from the enemy!) to replace each line break with

 DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci [line break] ALTER TABLE

You might need to type ALTER TABLE once at the beginning of the file. Then run the file as a batch SQL file against the database. Again, the details depend on what operating system, database server, etc. you have.

RLE

Average of ratings: -
In reply to Richard Enison

Re: HELP: UNICODE is required to be installed/enabled.

by Manuel de la Torre -
Thanks. But I took a more radical solution. I unistalled Moodle with Fantastico (many problems!) and installed it the old way!

Now everything is fine!!!

Thanks!
Average of ratings: -
In reply to Manuel de la Torre

Re: HELP: UNICODE is required to be installed/enabled.

by david onitzuka -
I actually ran the changes to all of the tables as well and it still complained.

I believe I may do the same and uninstall the Fantastico edition and install it manually myself as well.

Thanks for the input.

Average of ratings: -
In reply to david onitzuka

Re: HELP: UNICODE is required to be installed/enabled.

by Richard Enison -

DO,

What you have done is change the default character set and collation of the database and every table in it to Unicode. That's good but it is not enough. It is also necessary for the character set of every column in every table to be Unicode. When you create a new database from scratch, setting the default character set and collation of the database is enough, because after that, as tables are created, the default character set and collation of the database will be applied to those tables and the columns in them unless specified otherwise. That's what default means.

RLE

Average of ratings: -
In reply to Richard Enison

Re: HELP: UNICODE is required to be installed/enabled.

by david onitzuka -
Is there some easy command for doing this? To change the encoding and collation of all of the tables I had to feed the list of all of the tables (about 181 of them) through an awk script that generated the correct statements for each table.

But to now do it for all of the columns for each of these tables is quite onerous.

Does the Moodle installation already have some tool to automatically fix/correct this? The fact that Moodle needs utf8/utf8_unicode_ci, but does not set it at the database level where it needs it is a problem. That it runs into this repeatedly seems to suggest that it needs to:
  • set utf8/utf8_unicode_ci for the database at the beginning of the install
  • needs a tool that can go in and change everything automatically or on demand
Does this exist already?

Average of ratings: -
In reply to david onitzuka

Re: HELP: UNICODE is required to be installed/enabled.

by Richard Enison -

DO,

In a way there is. Normally there are two circumstances where the issue of Unicode comes up:

  1. A new installation. In this case, you create an empty database and give it the right default character set and collation, and that takes care of it.
  2. An upgrade. When upgrading from Moodle 1.6 or 1.7, there is a database migration script in the administration menu that you can run. By the time you get to 1.8 if you haven't migrated yet, it is too late. Unicode is optional for 1.6 and 1.7, unavailable before 1.6, but mandatory for 1.8. See http://docs.moodle.org/en/Database_migration; http://docs.moodle.org/en/Upgrading_to_Moodle_1.8#UTF-8_database_encoding

In a previous post, you spoke of doing a manual installation. If this was a new installation, the database should be empty so the problem should not come up. If the database is not empty, where did it come from?

RLE

Average of ratings: -
In reply to Richard Enison

Re: HELP: UNICODE is required to be installed/enabled.

by Kyle Claypool -
I recently upgraded from moodle 1.6 to 1.9 and got the UNICODE error myself, so I went back and upgraded to 1.7, ran the migration script, which informed me that the entire database is already UTF8-UNICODE. I then upgraded to 1.9 again and got the same error. I checked the tables in phpMyAdmin and they are indeed all "utf8_unicode_ci", so I'm stumped... any ideas?
Average of ratings: -
In reply to Kyle Claypool

Re: HELP: UNICODE is required to be installed/enabled.

by Richard Enison -
KC,

My previous posts in this thread were from before my involvement with MDL-11743. I now know much more about how Moodle tests for Unicode compliance with all four types of database.
  1. The fact that the migration script told you the database was already Unicode probably means that the Unicode flag was on. This flag is automatically turned on by the migration script when it is run. The fact that it was already on when you ran it might mean some well-meaning colleague of yours turned it on by putting

    $CFG.->unicodedb = true;

    (or 1 instead of true) in config.php at some point. It might no longer be there but it has been remembered by Moodle in the mdl_config table in the database (assuming your table prefix is the default mdl_, of course).
  2. What type of database do you have? I'm guessing MySQL, because that's the default. If not it is probably PostreSQL. It is certainly not MS SQL Server, because Moodle assumes such databases are always Unicode.
  3. Are you using a commercial web hosting service? Which one? I'm guessing bluehost or hostmonster. Because those are the ones where this problem kept coming up in the tracker issue I mentioned. If so, and if you are using MySQL, then I am disappointed, unless you are using an old download of 1.9. Because it means that the solution Anthony Borrow came up with in the tracker issue has not been implemented in 1.9. I'm pretty sure it is in the latest CVS version, but maybe not on the generic download page.
  4. If all of the above conditions are met, the way to be sure is to look at setuplib.php in the lib subfolder (subdirectory). Do a search for "variables like". If you see "show variables like 'character_set_database'", it means the fix has not been implemented. Insert the word "local" between "show" and "variables".
  5. If that doesn't help, your MySQL database really isn't Unicode compliant. If, as you say, all the tables are utf8, the default for the database itself needs to be changed. Run the queries

    ALTER DATABASE (your db name) CHARACTER SET utf8

    and

    ALTER DATABASE (your db name) COLLATE utf8_general_ci

    (or utf8_unicode_ci). Don't type the parentheses.
  6. If your database is PostgreSQL, and you are using one of the renegade web hosts mentioned above, contact tech support and ask them to make the database UTF-8.
RLE
Average of ratings: -
In reply to Richard Enison

Re: HELP: UNICODE is required to be installed/enabled.

by Kyle Claypool -
Richard,

Thanks for the detailed response. I think I solved the problem. I had a few Moodle plug-ins installed that must have fouled things up (Certificate and Exam I think), so I simply dumped the tables for those plug-ins. I was able to run the UTF8 migration script in my ver. 1.6 installation of Moodle, and after that the upgrade went smoothly.

Thanks again,
Kyle Claypool
Average of ratings: -
In reply to Richard Enison

Re: HELP: UNICODE is required to be installed/enabled.

by Paul Lane -
I just installed Moodle 1.9.5 (Build: 20090515) (first time - not an upgrade) on Hostmonster using SimpleScripts just last week. I get the same error message and I checked the language: it is set properly on the mysql database.

Apparently, the fix is still not available through my host.
Average of ratings: -
In reply to Richard Enison

Re: HELP: UNICODE is required to be installed/enabled.

by Greg Padberg -

Unicode db migration in 1.9.5?

I have a Moodle 1.9.5 installation that seems to be working OK. The initially installed version may have been earlier than Moodle 1.6.3 (my upgradelogs go at least that far back). This installation has been through several incremental updates / upgrades, i.e. Moodle always got updated shortly after any stable new release. I prefer to keep things simple, there are no modifications to this Moodle installation other than a custom theme.

Administration > Server > Environment yields the following info:

Server Checks report that “Unicode must be installed and enabled” (Status = “Check”), all other checks report “OK” for current stable Moodle installation.

Of course the following error also appears at the bottom of the page, “You must solve all the environmental problems (errors) found above before proceeding to install this Moodle version!”.

What I did so far, I noted that the 1.6 and 1.7 stable versions both had the utfdbmigrate.php script (md5sum was identical for both versions). I extracted the script and copied it to the current install and ran it from my web browser:

/moodle/admin/utfdbmigrate.php

The result is “unicode db migration has already been performed!”.

Could it be that the unicode db migration was already done but the suggested change to config.php was overlooked?

According to the config file docs,

http://docs.moodle.org/en/Configuration_file

My current config.php is missing in the following line:

$CFG->unicodedb = true; // Database is utf8

Should adding this line to config.php safely change the “unicode” server check to status = “OK”?

Hoping to get all the Environment Server Checks to “OK” before proceeding to upgrade this Moodle installation from 1.9.5 to 1.9.6 very soon! I also hope that no database changes are necessary to remedy this condition. Does this all seem to check out safely? Looking for some confirmation here before proceeding further…
Average of ratings: -
In reply to Manuel de la Torre

Re: HELP: UNICODE is required to be installed/enabled.

by John Anderson -
Just wanted to post that this even occurs when upgrading a 1.9.3 site to 1.9.4.

I looked at my DB and those tables are UTF8. What is the upgrade script, check actually not happy with, uh is my question?

1. the DB default is set to UTF8 (I understand, yes from above)
2. tables - yes?
3. table columns - yes?

Anything else?

And should not this just look at Moodle core tables and not add-on modules?

Also, I guess the upgrade script can not handle this conversion for us???
ALTER DATABASE moodle DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci

Thanks for ideas.
Average of ratings: -
In reply to John Anderson

Re: HELP: UNICODE is required to be installed/enabled.

by Steve Warnes -

Oh  Dear - this is all much to complicated for me.

I seem to have Moodle 1.9.3 installed.  I have updated it before with fantastico at my account at my provider.  Now moodle is insisting that it upgrades to 1.9.4 but tells me I have a problem with UTF 8 which it wants me to migrate my data to.  It tells me that I should do this in the admin page but the command is not there.  Gone with 1.7 I understand.

So help - how can I migrate this data in 1.9.3 - or what can I do?

Thanks Steve

Average of ratings: -
In reply to Steve Warnes

Re: HELP: UNICODE is required to be installed/enabled.

by Srecko Jovancevic -
ALTER TABLE `mdl_assignment` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_assignment_submissions` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_backup_config` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_backup_courses` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_backup_files` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_backup_ids` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_backup_log` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_blocks` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_cache_filters` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_cache_text` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_chat` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_chat_messages` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_chat_users` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_choice` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_choice_answers` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_config` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_course` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_course_categories` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_course_display` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_course_modules` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_course_sections` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_enrol_paypal` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_event` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_forum` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_forum_discussions` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_forum_posts` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_forum_queue` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_forum_ratings` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_forum_subscriptions` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_glossary` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_glossary_alias` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_glossary_categories` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_glossary_comments` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_glossary_entries` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_glossary_entries_categories` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_glossary_formats` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_glossary_ratings` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_groups` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_groups_members` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_journal` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_journal_entries` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_label` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_lesson` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_lesson_answers` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_lesson_attempts` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_lesson_grades` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_lesson_pages` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_log` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_log_display` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_modules` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_answers` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_attemptonlast_datasets` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_attempts` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_calculated` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_categories` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_dataset_definitions` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_dataset_items` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_grades` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_match` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_match_sub` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_multianswers` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_multichoice` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_numerical` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_numerical_units` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_question_datasets` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_question_grades` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_questions` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_randomsamatch` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_responses` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_shortanswer` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_quiz_truefalse` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_resource` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_scale` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_scorm` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_scorm_sco_users` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_scorm_scoes` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_survey` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_survey_analysis` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_survey_answers` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_survey_questions` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_user` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_user_admins` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_user_coursecreators` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_user_preferences` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_user_students` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_user_teachers` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_wiki` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_wiki_entries` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_wiki_pages` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_workshop` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_workshop_assessments` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_workshop_comments` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_workshop_elements` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_workshop_grades` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_workshop_rubrics` CHARSET utf8, COLLATE utf8_general_ci;
ALTER TABLE `mdl_workshop_submissions` CHARSET utf8, COLLATE utf8_general_ci;
Average of ratings: -
In reply to Srecko Jovancevic

Re: HELP: UNICODE is required to be installed/enabled.

by Srecko Jovancevic -
yes but the default collate must be th same on the DB server
Average of ratings: -
In reply to Srecko Jovancevic

Re: HELP: UNICODE is required to be installed/enabled.

by Karen Erickson -

I am having the same error code and opened this via my server, it was not an update.  Help!

It is required that you store all your data in Unicode format (UTF-8). New installations must be performed into databases that have their default character set as Unicode. If you are upgrading, you should perform the UTF-8 migration process (see the Admin page).

Average of ratings: -