1

We have a website which is dealing with Chinese characters and was hosted on AWS.

Here I can save Chinese characters in database without any problem.

Now we move to Google Cloud and I am facing issue saving Chinese characters in database.

They display as 一地兩檢

I am following all rules like "column should be utf8-unicode-ci" and "database connection as utf8".

It is working fine on localhost.

Any Idea what can be problem ?

Thanks.

Vivek Tankaria
  • 1,301
  • 2
  • 15
  • 35
Rich5757
  • 155
  • 2
  • 12

2 Answers2

2

If the data (column) in the database holds (similar) UTF8-encoded data in both cases and the code/platform which handles the data in the web-page is the same (meaning not python 2 vs python 3 for example), the difference might be the current local setting, either of the Google server (environment-variables), the SQL-client (UTF8-settings) or the php-settings. Lets start with the sql-client: Try to run the php - function

mysqli_character_set_name()

to get the encoding. If it is not UTF-8 then set it with 

mysqli_set_charset('utf8')

If this is not working ensure the php-html stuff by setting the charset in the META html-tag to utf-8

charset=utf-8

and enforce it with

declare(encoding='utf8')
Gunnar Sigfusson
  • 81
  • 8
  • I am using php and database version is 5.6 – Rich5757 Dec 28 '17 at 11:47
  • Ok, I assume this environment here [link] (https://cloud.google.com/php/getting-started/using-cloud-sql-with-mysql) Start with comparing the supported database locals [link] (https://dev.mysql.com/doc/refman/5.5/en/charset-unicode.html) . If this is ok then the problem could be in the client, check local settings on your Linux (Google) platform to your previous platform [link] (https://www.cyberciti.biz/faq/how-to-set-locales-i18n-on-a-linux-unix/) Note: If you are using the MySQL database version 5.x check that your (ODBC?) client is also 5.x (or supports that) – Gunnar Sigfusson Dec 28 '17 at 13:11
  • I tried above link , but not helping. All thing utf8 – Rich5757 Dec 28 '17 at 13:40
  • 1
    I found one thing here , may be it will help. When I run direct insert query using sqlyog I can insert Chinese characters but they are not inserting properly when using php – Rich5757 Dec 28 '17 at 17:38
  • Yes we are getting nearer. The problem is either in the web page META settings or in the settings of the SQL-client php is using. Lets start with the sql-client: Try to run the php - function "mysql_client_encoding" to get the encoding. If it is not UTF-8 then set it with "mysql_set_charset( 'utf8' )" If this is not working ensure the php-html stuff by " " and enforce it with "declare(encoding='utf8')" – Gunnar Sigfusson Dec 28 '17 at 19:18
  • Hi , Gunnar , Its working now. You can post a good answer based on above discussion and I will mark this as accepted answer so it can help others also. Thanks again. – Rich5757 Dec 29 '17 at 08:53
  • Ok great, you pointed me in right direction so i added a hat too (can do that now). I will edit the original answer – Gunnar Sigfusson Dec 29 '17 at 12:23
  • 1
    Do _not_ use PHP's `mysql_*` interface; use either `PDO` or `mysqli_*`. – Rick James Dec 29 '17 at 19:56
  • 1
    @GunnarSigfusson - Please do not use any conversion routines. They are likely to make things worse, in which case it will be much more difficult to unravel things. – Rick James Dec 30 '17 at 14:31
  • Yes it is not necessary here to use the iconv. I was not familiar with the lot of stuff php offers. My world is more in the C/Python and unfortunately I need it in my work as I have to save messages of different encodings in same column in database and fewer convertion options in C, but it is not necessary here. I also will re-edit the answer to mysqli_* and remove iconv - comments – Gunnar Sigfusson Dec 30 '17 at 17:32
  • 1
    @GunnarSigfusson - thanks for updating. One more thing: `mysqli_client_encoding()` has been removed from PHP and replaced by `mysqli_character_set_name()`. Ref: http://php.net/manual/en/function.mysqli-client-encoding.php – Rick James Dec 30 '17 at 23:24
  • @RickJames Updated the answer with set_name function thanks for the hint. – Gunnar Sigfusson Jan 01 '18 at 15:51
1

Looks like you have latin1 somewhere in the processing.

一地兩檢 is "Mojibake" for 一地兩檢

See Mojibake in Trouble with UTF-8 characters; what I see is not what I stored

Some Chinese characters take 4 bytes, not just 3 bytes. So, I recommend you use utf8mb4, not simply utf8.

Rick James
  • 135,179
  • 13
  • 127
  • 222