0

I'm using django and tastypie for my api. the db is MySQL.

After a few days of fiddling around i managed to store emoji icons in the db using utf8mb4 character set.

When querying the db directly from the console (on a mac), i see the emoji fine, but when pulling them from the api (for example using the browser), the json shows question marks. This leads me to believe the issue is not with the db but with django/tastypie db connection.

How do i go about it?

itayab81
  • 83
  • 4

2 Answers2

1

The solution is in DJango settings:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {'charset': 'utf8mb4'},
        (...)
itayab81
  • 83
  • 4
0

What if you save the JSON dump to a text file and open it with a UTF8 text editor? Will the emoji re-appear?

I suspect that, when displaying the JSON in the browser, the browser falls back to Latin-1 encoding, thus ending up with question marks. There might be a setting in your browser to set the encoding.

Check the "Content-Type" header that comes from your server. Does it say application/json; encoding=utf-8?

BastiBen
  • 19,679
  • 11
  • 56
  • 86
  • saving as text did not help. the content type is "application/json" but doesn't say "encoding=utf-8" – itayab81 Mar 03 '13 at 20:50
  • So that is likely the problem. Emoji are extended Unicode/UTF-8 characters and won't display unless the encoding is specified correctly everywhere. Normally I'd expect tastypie to be utf-8 though. Do they show up in django admin? – BastiBen Mar 04 '13 at 08:51
  • after following your lead i stumbled upon this: https://github.com/toastdriven/django-tastypie/issues/717 it's a bug fix that removes the charset from the content type if it's json – itayab81 Mar 04 '13 at 19:28