Correct character encoding

Question

I'm was phrasing a file and found a error with the file name while saving to mysql database.

Original file name looks something like Thor - Thor：The Dark World but after phrasing and saving the name to mysql through PHP, the string turns to Marvels Thorï¼šthe Dark World.

It displays the same even when I ECHO the PHP variable that holds the string so this has nothing to do with mysql.

where am I doing wrong? Any suggestions?

$di = new RecursiveDirectoryIterator('/var/www/example/data/');
foreach (new RecursiveIteratorIterator($di) as $filename => $file) {


        echo $file->getFilename(); //displays Marvels Thorï¼šthe Dark World

        $data[] = array('file'=>array('name'=>$file->getFilename(),'path'=>$filename)); 


}

By the way, if I store `Marvels Thorï¼šthe Dark World` in a file in ISO-8859-1, and then I open it in UTF-8 it reads `Thor - Thor：The Dark World`. — Theraot, Dec 07 '13 at 06:19

Theraot · Accepted Answer · 2013-12-07T06:13:37.700

There are two things to do...

1) Set the encoding the HTTP headers. for example:

header('Content-Type: text/html; charset=UTF-8')

2) Set the encoding in the HTML headers. for example:

echo '<!DOCTYPE hmtl>';
echo '<head>';
echo '<meta charset="utf-8">';
echo '</head>';

Or pre HTML5:

echo '<meta http-equiv="Content-Type" content="text/html; charset=utf-8">'

You may also want to...

3) Tell PHP to use a particular encoding, for example:

ini_set('default_charset', 'UTF-8');
mb_internal_encoding('UTF-8');

4) Whenever possible match the encoding of the database with the one of PHP...

For that you may try to change the encoding of the database from PHP. If using PDO set it in the connection string (aka DSN), if using mysqli you can use mysqli::set_charset.

Otherwise use a query:

$connection->query('set charset UTF8');

... Or convert the strings (see iconv at php.net).

5) You should know your encodings. It is meaning less to have strings and have no clue what encoding they are in... in particular if you are reading from a file or similar source. Make sure to store your files in a known encoding.

Other alternatives include using HTMLEntities. On doing that:

The functions htmlspecialchars and htmlentities will not convert all the characters of your encoding, only those "dangerous" (htmlspecialchars) or that have html equivalent named entities (htmlentities). If you want more control you will need to use mb_encode_numericentity instead.

This is so great and worked perfectly...! Thank you for your answer and time. — Naveen Gamage, Dec 07 '13 at 12:21
I like the way you said `there's two things to do` and then put 5 different steps. — , Dec 07 '13 at 22:41

Lucas · Answer 2 · 2013-12-07T06:07:05.523

The problem is that the file name consists of a unicode character (which may look like a colon), but cannot be displayed under other encodings (for example, ISO-8859-1, or the common Western encoding).

What you have to do, is find the offending character (perhaps a colon, as I know it), delete it, and type it in again with your keyboard (and if it is in a program like Notepad, make sure the encoding is not ANSI, but is UTF-8), and that should fix the problem.

Edit: you may also need the following at the top of your PHP document (as Hussain said):

ini_set('default_charset', 'UTF-8');
mb_internal_encoding('UTF-8');
mysql_set_charset('utf8'); // for mysql
// mysqli_set_charset($conn, 'utf8'); // for mysqli procedural]
// $mysqli->set_charset('utf8'); // for mysqli object-oriented

Also, for the output, you may need:

header('Content-Type: text/html; charset=utf-8');

That may help display the character.

He might also have to do `query("SET NAMES utf8");` before saving it in `mysql` right? — Hussain, Dec 07 '13 at 05:57

Correct character encoding

2 Answers2