corrently printing strange characters

Question

For the life of me, I cant figure out what I am doing wrong

import urllib
import csv

with open("mydb.txt", 'rb') as f:
    readr = csv.reader(f, delimiter = ",", quotechar="'")
    for row in readr:
        mylist = []
        for i in row:
            code=urllib.unquote(i)
            mylist.append(code)
        print mylist

the problem is I keep getting things like:

['S\xc3\xa3o Desid\xc3\xa9rio', 'BA', 'Convencional', '1759', '-12.52332', '-45.59509']

What is this 'S\xc3\xa3o Desid\xc3\xa9rio' ? it should be São Desidário. How can I fix it?

You are printing the *list object*, which shows string representations. These representations are for UTF-8 encoded byte strings and are entirely correct. — Martijn Pieters, May 15 '14 at 17:47
@relima I have shown how to decode it in my answer, it's not too difficult. — anon582847382, May 15 '14 at 17:49
@Martijn Pieters: you are right! I was printing a list object.. and that was my mistake. thanks. — relima, May 15 '14 at 18:11

score 0 · Answer 1 · answered May 15 '14 at 20:49

You are printing the list object instead of its members. Trying using str.join to format the list to your liking.
You should consider the "unicode sandwich" approach ("bytes on the outside, unicode on the inside"). Convert all of your input to unicode immediately upon input, and convert it to encoded bytes on output.

This program might suit you:

import urllib
import csv

with open("mydb.txt", 'rb') as f:
    readr = csv.reader(f, delimiter = ",", quotechar="'")
    for row in readr:
        mylist = []
        for i in row:
            i = i.decode('utf-8')
            code=urllib.unquote(i)
            mylist.append(code)
            print type(code),code
        print u','.join(mylist).encode('utf-8')

corrently printing strange characters

1 Answers1