Django + sqlite + Unicode

Question

I faced a problem with Unicode strings while adding new records to a sqlite database through the admin site.

class Translation(BaseModel):
  .....
  translation = models.CharField(max_length=100)

When I try to insert a word like 'été' an error occurs:

**UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 0: ordinal not in range(128)**

Update: Added traceback: http://pastebin.com/yLYFNDGB

does this question http://stackoverflow.com/questions/2392732/sqlite-python-unicode-and-non-utf-data help? — Andrew Cox, Jul 10 '12 at 14:45
@AndrewCox: I believe that Django already takes care of all that, see https://docs.djangoproject.com/en/dev/ref/unicode/. — Martijn Pieters, Jul 10 '12 at 14:48
Please include the full traceback you get so we can pinpoint the exact cause. — Martijn Pieters, Jul 10 '12 at 14:48

score 5 · Accepted Answer · edited Jul 12 '12 at 18:12

5

I found a solution. Actually, the problem was not in Django or sqlite. The issue was with the unicode() method.

Previously it was:

def __unicode__(self):
    return "{} ({})".format(self.translation, self.word.lang.abbr)

After an obvious fix, the problem is gone:

def __unicode__(self):
    return u"{} ({})".format(self.translation, self.word.lang.abbr)

edited Jul 12 '12 at 18:12

dda

6,030
2
25
34

answered Jul 10 '12 at 15:22

Alex G.P.

9,609
6
46
81

could you mark the question as answered? it's okay if you answered it yourself. – darkphoenix Jul 10 '12 at 16:55

score 0 · Answer 2 · answered Jul 10 '12 at 15:06

When you assign é as a string to a variable, it actually takes it as xe9.

Like if you go to your python command line editor and write following code:

>>> a = u'café'
>>> a

and press enter, the é character is replaced by \xe9 as illustrated as follows:

>>> u'caf\xe9'

on the other hand, if you write this with print statement, you would get what you initially assigned, i.e:

>>> print a
café

So, after making this point, the solution to your problem is using utf8 encoding to get rid of \xe9. So you can encode your string in the following way:

>>> string_for_output = yourstring-with-é.encode('utf8', 'replace')

For more about this topic, you may consult THIS URL . It discusses your problem under the heading "Frustration #3: Inconsistent treatment of output"

Hope this helps.

My question is not about python+unicode, but about django+unicode. — Alex G.P., Jul 10 '12 at 15:13

Django + sqlite + Unicode

2 Answers2

Linked