import utf-8 csv in python3.x - special german characters

Question

I have been trying to import a csv file containing special characters (ä ö ü)

in python 2.x all special characters where automatically encoded without need of specifying econding attribute in the open command.

I can´t figure out how to get this to work in python 3.x

import csv

f = open('sample_1.csv', 'rU', encoding='utf-8')
csv_f = csv.reader(f, delimiter=';')

bla = list(csv_f)
print(type(bla))

print(bla[0])
print(bla[1])
print(bla[2])
print()
print(bla[3])

Console output (Sublime Build python3)

<class 'list'>
['\ufeffCat1', 'SEO Meta Text']
['Damen', 'Damen----']
['Damen', 'Damen-Accessoires-Beauty-Geschenk-Sets-']

Traceback (most recent call last):
  File "/Users/xxx/importer_tree.py", line 13, in <module>
    print(bla[3])
UnicodeEncodeError: 'ascii' codec can't encode character '\xf6' in position 37: ordinal not in range(128)

input sample_1.csv (excel file saved as utf-8 csv)

Cat1;SEO Meta Text
Damen;Damen----
Damen;Damen-Accessoires-Beauty-Geschenk-Sets-
Damen;Damen-Accessoires-Beauty-Körperpflege-
Männer;Männer-Sport-Sportschuhe-Trekkingsandalen-
Männer;Männer-Sport-Sportschuhe-Wanderschuhe-
Männer;Männer-Sport-Sportschuhe--

is this only an output format issue or am I also importing the data wrongly?
how can I print out "Männer"?

thank you for your help/guidance!

The problem seems to be your *terminal* isn't configured to print unicode — juanpa.arrivillaga, Mar 29 '18 at 01:45
So then definitely that's the problem. Is your local set for UTF-8? — juanpa.arrivillaga, Mar 29 '18 at 08:14
thank you! in the terminal my stdout.encoding is set to **UTF-8** in fact the csv is imported correctly when launched through terminal. only issue is when launching the build through **Sublime** (US-ASCII), is it possible to switch to utf-8 without messing up environment variables? — Shahin Ghannadian, Mar 29 '18 at 08:39

score 0 · Accepted Answer · answered Mar 29 '18 at 09:17

thank you to juanpa-arrivillaga and to this answer: https://stackoverflow.com/a/44088439/9059135

Issue is due to my Sublime settings: sys.stdout.encoding returns US-ASCII

in the terminal same command returns UTF-8

Setting up the Build system in Sublime properly will solve the issue

import utf-8 csv in python3.x - special german characters

1 Answers1