Can't import CSV files including Chinese character

Can't import CSV files including Chinese character

by spoony liu -
Number of replies: 5

Can't import CSV files including Chinese character for user and database when the appearance is Chinese(zh_cn)，but the same CSV files work well when the appearance is English(en).

Position:

Dashboard/Site administration/Users/Accounts/Upload user

Dashboard/My courses/Moodle/数据库/上报学生信息/Import entries/Add entry

The old version 2.8.5+has no problem above.

The current version is 3.6.4+.


Average of ratings: -
In reply to spoony liu

Re: Can't import CSV files including Chinese character

by koen roggemans -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Translators

This should be tested on the latest version.

Can you do that or attach a file here that doesn't work?

Bugs can be reported on https://tracker.moodle.org

Average of ratings: -
In reply to koen roggemans

回复: Re: Can't import CSV files including Chinese character

by spoony liu -
I have tested the latest moodle( 3.7+ Build: 20190704). The problem does exist. Here is a file of user that I tested. It works well on  2.8.5+ that  I have been using for a long time.
Average of ratings: -
In reply to spoony liu

Re: 回复: Re: Can't import CSV files including Chinese character

by koen roggemans -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Translators

The document is ISO-8859-1 encoded. It works when uploading it and changing the document encoding to ISO-8859-1, but what comes out, doesn't look Chinese.

I don't know how to get back to Chinese characters starting from your document - it might be not possible.

Anyway, somewhere in the process of creating your document, you mixed up encodings, with this as a result. I think it's better to use UTF-8 for Chinese.

I made an example document. I don't know any Chinese - I just got some characters from a Chinese page somewhere, so the content of my file is complete nonsense and hopefully not offending smile


Average of ratings: Useful (2)
In reply to koen roggemans

Re: 回复: Re: Can't import CSV files including Chinese character

by Germán Valero -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators
I have often used Notepad++ to change the encoding of a file. It is fast, easy and free ;)
Average of ratings: -
In reply to Germán Valero

Re: 回复: Re: Can't import CSV files including Chinese character

by koen roggemans -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Translators
For this case, the current encoding is wrong - it is not the correct encoding that goes with the bytes in the file. So if you change the encoding of this file the correct way, like using Notepad++ or any other tool, the characters will never show up correctly - they will show the same wrong characters in a different encoding.
I don't know if it is possible to figure out what went wrong and revert that. I think in this case, the only thing you can do is start over and make sure you know during the whole process of creating the file which encoding you use and make sure that that encoding supports Chinese.
ISO_8859-1 doesn't support Chinese - it is a single byte encoding, only supporting 191 characters. There have been several encoding standards for Chinese, but not this one. So something went wrong in the creation of this file: the original encoding got lost and replaced wrongly by ISO_8859-1, thereby damaging the file.
It is weird that this works on a 2.8. A lot more weird then that it doesn't work on a 3.6.
Average of ratings: Useful (1)