Special Characters Corruption During Database Dump and Import

Special Characters Corruption During Database Dump and Import

by Shoun Paul -
Number of replies: 1

Hello Moodle Community,

I am encountering an issue with special characters being corrupted during the database dump and import process. Here's the scenario:

  1. Database Information:

    • I am using MariaDB version 10.0.35.
    • The character set is utf8mb4_unicode_ci.
    • Moodle version 3.5 stable
  2. Issue Details:

    • We have a question bank created by one of our Chinese teachers in PINYIN language. The questions contain special characters with signs above them, such as:

      • wǎnān, wánān, wǎnán
      • búcuò, bǔcuò, būcuò
    • After exporting the database and importing it back, these characters appear corrupted. For example:

      • Original: wǎnān → Corrupted: w├ín─ün
      • Original: búcuò → Corrupted: b┼½cu├▓
  3. Observations:

    • If we manually rewrite the characters in Moodle, they display correctly.
    • During repeated imports, some questions appear correct, but the issue persists with most of them. The process used for exporting and importing remains the same in all cases.
  4. Steps Taken:

    • The database character set and collation are correctly set to utf8mb4_unicode_ci.
  5. Export and Import Commands Used:

    • Export:
      $Command = """$MySQLDumpPath"" --quick --extended-insert --routines --triggers --no-set-names --single-transaction --create-options --default-character-set=utf8mb4 --flush-logs --all-tablespaces --complete-insert --lock-tables -u$DBUser -p$DBPass $DBName > `"$BackupFile`""
          cmd.exe /c $Command
    • Import:
      mysql -u root -p Moodle_DB < "C:\MoodleDB_Dump.sql"  
      

Request for Guidance:

  • How can I ensure that the special characters are preserved correctly during the export and import process?
  • Are there additional settings or tools I should be using with mysqldump or during the import?

Your assistance in resolving this issue will be greatly appreciated. Thank you!

Average of ratings: -
In reply to Shoun Paul

Re: Special Characters Corruption During Database Dump and Import

by William Lu -
Moving to the General help forum...