0

We have a live web services iPhone app where the content from the server is read from a MySQL table (Type: MyISAM; Collation: latin1_swedish_ci) & then sent through PHP via XML.

Everything was running fine until yesterday we needed to support ◉ character.

In the XML line in our PHP we changed

$xml_output = "<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>\n";

to $xml_output = "<?xml version=\"1.0\"?>\n";

And voila, along with that character, the app started supporting Emoji & International language!

But the problem is.. now when the content contains some particular characters, the app crashes, and we are unable to detect those characters.

As an example: When the MySQL field has this 'â¤ðŸ’˜ðŸ’UserFoo🌹â™' string, the app crashes. The weird characters in this example are supposedly some Emoji Characters.

I believe this is some encoding related issue, but have no idea how to set it up alright - from insertion into table using PHP & then fetching through XML.

How do I set the whole process? Also, it would be great if you could point to some resource for beginners that describes the whole thing with emoji support.

PS: I read this question & followed the suggestion, but couldn't fix yet.

Community
  • 1
  • 1
BufferStack
  • 549
  • 9
  • 20
  • What *exactly* crashes? If it's the app, this can be totally unrelated to PHP. – hakre Mar 24 '12 at 14:13
  • The app crashes.. but it is related to characters as if I just revert the above mentioned change in the xml line, the crash doesn't happen but the characters display weird. – BufferStack Mar 24 '12 at 14:53
  • I would restore the encoding attribute, maybe try: $xml_output = "\n"; – Andrew Kandels Mar 24 '12 at 14:55
  • @BufferStack: That's a clear sign that the app crash is remotely exploitable, but not that the remote data is the cause of the crash. A crash can only be internal, not external. The crash only makes a flaw in your app visible. You should now concentrate to fix the flaw first before doing anything else. Glad you found this and you already know how you can reproduce the crash! – hakre Mar 24 '12 at 15:12
  • @hakre: Thanks, but the crash is related to the above mentioned problem. I we can have encoding issue resolved, then crash also gets fixed. I cannot fathom what change shall I do on the app side?! – BufferStack Mar 24 '12 at 15:27
  • @AndrewKandels - already tried that buddy - no help! – BufferStack Mar 24 '12 at 15:28
  • A crash is related to the flaw, and strictly spoken not to the mentioned problematic data that triggers the flaw. Fix the flaw first. You will notice that this also solves the mentioned problem. But you actually fix something instead of trying. Trust me, fixing the flaw will cure the problem as well. Not fixing it will leave it open to the future. – hakre Mar 24 '12 at 15:29

2 Answers2

0

You should also know that most of the emoji characters are encoded using a 4 byte UTF8 codepoint which cannot be stored in MySQL unless you upgrade to version 5.5 and convert your tables to the utf8mb4 character set.

Reference: How to insert utf-8 mb4 character(emoji in ios5) in mysql?

I am guessing that your app is crashing because the XML parser has encountered an invalid character. The invalid character probably creeped in because of the encoding conversions that you are doing. If you convert everything server side to UTF8, it will probably fix the problem, but if you accept user input then you will also need to sanitize the input to make sure that it is valid UTF8.

Community
  • 1
  • 1
Jake
  • 1,135
  • 1
  • 12
  • 26
0

I think what you should first try doing is changing your DB and table encoding to "utf8_general_ci".

You should then run SET NAMES 'utf8' on your MySQL connection. This is important so that all client communication happens in unicode.

And then when you write your XML, set <?xml version="1.0" encoding="UTF-8"?>

If none of the above works, I will suggest that you also review your iPhone code and configs to ensure that special characters are supported.

Hope it helps!

Abhay
  • 6,545
  • 2
  • 22
  • 17
  • Additionally, this SO post might help - http://stackoverflow.com/questions/1705315/questions-about-iphone-emoji-and-web-pages – Abhay Mar 24 '12 at 15:54