Unexpected behaviour while decoding strings in PHP (from AJAX POST call)

Question

I have some javascript that sends data in JSON through POST format to a PHP script.

Everything works fine with "usual" characters, but I see inconsistencies when using, for example, vowels with accents such as "à". I would like to ask if anyone has suggestions on how to fix this.

This is the Javascript:

$.ajax({
        contentType: 'application/json',
        data: JSON.stringify({
            "action": params.action,
            "username": params.username,
            "page": params.page,
        }),
        processData: false,
        //dataType: 'json',
        url: "/w/ImolaCustom/SudoAutoedit.php",
        type: 'POST',
        success: function(data) { 
            ...
        }
    });

On the PHP side of things I do this:

$theData = json_decode(file_get_contents('php://input')), true);

The problem presents itself if I send something like:

params.page = "Società sportiva Bridge";

as $theData['page'] becomes "Societ\xc3\xa0 sportiva Bridge"

If I use utf8_decode($theData['page']) (or if I use it on the string passed from php://input before json_decoding it I get "Societ\xe0 sportiva Bridge" instead.

I tried different conversion functions like iconv(), mb_convert_variables() and mb_convert_encoding() to convert from UTF-8 to ISO-8859-1 with the same results as above.

I also tried encoding the string client-side with encodeURIComponent() or escape(). PHP receives the correct string (respectively "Societ%C3%A0%20sportiva%20Bridge" and "Societ%E0%20sportiva%20Bridge"), but after decoding it with rawurldecode() I still get "Societ\xc3\xa0 sportiva Bridge" and "Societ\xe0 sportiva Bridge" respectively.

Both files are on a CentOS machine and are saved with EOL Conversion in UNIX Mode and with Charset Encoding set to UTF-8 (editor is notepad++).

Please try `contentType: 'application/json; charset=utf-8'` ! UTF-8 is supposed to handle all accents. — Ismail RBOUH, Jul 27 '16 at 13:53
@IsmailRBOUH Just tried, no change, still getting the same results — Night861, Jul 27 '16 at 13:56
Can you call `console.log(JSON.stringify({ "action": params.action, "username": params.username, "page": params.page, }))` to determine if its backend or frontend issue? — RaV, Jul 27 '16 at 14:04
@RaV Hey, I had tried printing the client stuff and forgot to mention it, here you go anyways: {"action":"toggleFav","username":"Admin","page":"Società sportiva Bridge"} — Night861, Jul 27 '16 at 14:15
http://stackoverflow.com/questions/1805802/php-convert-unicode-codepoint-to-utf-8 - try this maybe? — RaV, Jul 27 '16 at 14:20

Ismail RBOUH · Accepted Answer · 2016-07-27T14:54:16.660

1

Please try this:

$content = file_get_contents('php://input');
$content = mb_convert_encoding($content, 'UTF-8',
          mb_detect_encoding($content, 'UTF-8, ISO-8859-1', true));

$theData = json_decode($content, true);

OR:

$content = file_get_contents('php://input');
$content = html_entity_decode(mb_convert_encoding($content, 'HTML-ENTITIES', "UTF-8"));

$theData = json_decode($content, true);

I hope this will help you.

edited Jul 27 '16 at 14:54

answered Jul 27 '16 at 14:05

Ismail RBOUH

10,292
2
24
36

Hey. I've done as you suggested and I still get "Societ\xc3\xa0 sportiva Bridge". I am starting to wonder if the problem could be related to some server-side setting. – Night861 Jul 27 '16 at 14:18
My bad ! in the answer I forgot to assign the decoded string to `$content`. Please try the both solutions in the answer. I hope this will help you solve the problem ! – Ismail RBOUH Jul 27 '16 at 14:56

Unexpected behaviour while decoding strings in PHP (from AJAX POST call)

1 Answers1