UnicodeEncodeError when trying to convert Django models to XML

Question

I found a python program: Export Django database to xml file that converts django models to a xml representation. I get these errors when trying to run the program. My models contain some text written in French.

Traceback (most recent call last):
  File "xml_export.py", line 71, in <module>
  writer.content(value)
File "xml_export.py", line 41, in content
  self.output += str(text)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 3:
ordinal not in range(128)

Tim Pietzcker · Accepted Answer · 2011-01-27T14:39:34.910

7

It looks like your variable text contains a non-ASCII string.

See:

>>> mystring = u"élève"
>>> mystring
u'\xe9l\xe8ve'
>>> str(mystring)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 0: ordinal not in range(128)

So, you first need to encode your string into UTF-8:

>>> str(mystring.encode("utf-8"))
'\xc3\xa9l\xc3\xa8ve'

Or, if (as the comments show) text may contain other variable types besides strings, use

self.output += unicode(mystring).encode("utf-8")

edited Jan 27 '11 at 14:39

answered Jan 27 '11 at 14:01

Tim Pietzcker

328,213
58
503
561

If I do that I get: Traceback (most recent call last): File "xml_export.py", line 71, in writer.content(value) File "xml_export.py", line 41, in content self.output += str(text.encode("utf-8")) AttributeError: 'int' object has no attribute 'encode' – Seitaridis Jan 27 '11 at 14:06
That's strange. It means that `text` holds an integer variable. This in turn makes it hard to figure out why there would be an `é` in `text` unless that variable is changing its type along the way (which would be legal in Python) but makes this situation more difficult. You'll need to provide more info - where does `text` come from? What are you doing with it? – Tim Pietzcker Jan 27 '11 at 14:09
The text data comes from the fields of the database, so the value of a field can contain also numbers. – Seitaridis Jan 27 '11 at 14:18
I tried unicode(text). This solves the problem in the content method, but then I have another error in the save method at the fp.write(self.output) line. The error I get is this: Traceback (most recent call last): File "xml_export.py", line 77, in writer.save("export.xml") File "xml_export.py", line 48, in save fp.write(self.output) UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 404 : ordinal not in range(128) – Seitaridis Jan 27 '11 at 14:25

Daniel Roseman · Answer 2 · 2011-01-27T14:37:49.790

3

Seriously, don't use the linked code. It's terrible, and appears to have been written by someone with absolutely no knowledge of unicode, character encodings, or even how to build up an XML document. String concatentation? Really?

Just don't use it.

edited Jan 27 '11 at 14:37

answered Jan 27 '11 at 14:30

Daniel Roseman

588,541
66
880
895

score 0 · Answer 3 · answered Jan 27 '11 at 14:13

0

Did you tried to use the built-in command :

./manage.py dumpdata --format xml

The way you used unicode in u'élève' is ok, so this should work (normalement...).

answered Jan 27 '11 at 14:13

Dominique Guardiola

3,431
2
22
22

I think this command does the same thing as this from django.core import serializers data = serializers.serialize("xml", SomeModel.objects.all()). I need to have the format from this question http://stackoverflow.com/questions/4757402/convert-python-object-to-xml-representation – Seitaridis Jan 27 '11 at 14:21
ben alors t'as qu'à utiliser la réponse! si tu déclare bien ton xml avec un encodage utf-8 au début, aucun souci! – Dominique Guardiola Jan 29 '11 at 14:23

UnicodeEncodeError when trying to convert Django models to XML

3 Answers3

Linked