Unicode Error on Python

Question

So I'm trying to make a table with a csv table that has unicode characters in it:

with open('test1.csv', 'w') as csvfile:
    writer = csv.writer(csvfile)
    [writer.writerow(r) for r in table]

I get this error every time i try to run my program, though:

UnicodeEncodeError: 'ascii' codec can't encode characters in position 8-10: ordinal not in range(128)

How do I fix this?

What version of Python are you using? If python 3, did you try openning the file in the enoding in which your `row` is with `encoding=''` argument to `open()` ? — Anand S Kumar, Sep 30 '15 at 15:31
possible duplicate of [Python: Convert Unicode to ASCII without errors for CSV file](http://stackoverflow.com/questions/4650639/python-convert-unicode-to-ascii-without-errors-for-csv-file) — Hacketo, Sep 30 '15 at 15:31

Mazdak · Answer 1 · 2015-09-30T15:59:39.870

2

First of all you don't need to use a list comprehension for write in csv file, secondly if you are using python 2.X you can use codecs module to open your file with a proper encoding and if you are using python 3.X you can use encoding argument in open function.

Also note that since the write method used your default encoding if still you got unicode error you can use str.encode() method in write method.

Python 2.X:

import codecs
with codecs.open(filename, 'w', encoding='utf-8') as csvfile:
     writer = csv.writer(csvfile)
     for r in table:
         writer.writerow(r.encode('utf-8'))

Python 3.X :

with open(filename, 'wb', encoding='utf-8') as csvfile:
     writer = csv.writer(csvfile)
     for r in table:
         writer.writerow(r.encode('utf-8'))

edited Sep 30 '15 at 15:59

answered Sep 30 '15 at 15:34

Mazdak

105,000
18
159
188

I get this error when I use your code: UnicodeEncodeError: 'ascii' codec can't encode characters in position 8-10: ordinal not in range(128) – MarkJ Sep 30 '15 at 15:54
I'm getting this error now: AttributeError: 'list' object has no attribute 'encode' – MarkJ Sep 30 '15 at 16:24
@K.B Is your table contains lists? – Mazdak Sep 30 '15 at 16:28
Yeah, the element table is full of lists – MarkJ Oct 01 '15 at 10:54

score 2 · Accepted Answer · answered Sep 30 '15 at 16:25

Assuming you're using Python2:

with open('test1.csv', 'wb') as csvfile:
    writer = csv.writer(csvfile)
    for r in table:
        writer.writerow([x.encode('utf-8') for x in r])

Of course, when you open the csv file, you will also need to decode it using the same encoding:

with open('test1.csv') as csvfile:
    reader = csv.reader(csvfile.decode('utf-8'))

(NB: if you used Python3, none of this would be necessary - your original example would work fine as it is).

Unicode Error on Python

2 Answers2