Cakephp display symbols from table and python encode('unicode-escape') related issue

Question

In my table, I have a row like this:

Amazing ...

When I try do display it in my view, it show this:

Amazing ... ???

In the head of the html page, I have well the tag

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

In core.php I have:

Configure::write('App.encoding', 'UTF-8');

In my database.php, I have:

public $default = array(
    'datasource' => 'Database/Mysql',
    'persistent' => false,
    'host' => 'localhost',
    'login' => 'root',
    'password' => 'xxx',
    'database' => 'xxx',
    'prefix' => '',
    'encoding' => 'utf8',
);

I am converting a current python script in php and I can see this code:

'comment_text': row[2].encode('unicode-escape'),

I tried for find the equivalent for encode('unicode-escape') in php but nothing found.

Do I need to use a similar function for my php display or I don't need to use this function equivalent and something is wrong with my encoding setup ?

Answer 1

I also had same problem before. The thing is the utf8 encoding only supports three bytes per character. You can read detail here

MySQL’s utf8 isn’t UTF-8. So, you can't save some char and emoji and sometime it may cut off your text.

What I did is I applied utf8mb4 to all table and schema.

E.g

`ALTER TABLE your_table CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;`

`ALTER SCHEMA `your_schema`  DEFAULT CHARACTER SET utf8mb4  DEFAULT COLLATE utf8mb4_unicode_ci;`

After onward, happy to saved emoji character as well :).