unicode datas of a dataframe to strings

Question

I have some troubles with a dataframe obtained from reading a xls file. Every data on such dataframe has the type 'unicode' and I can't do anything with this. I wanna change it to str values. Also, iff possible, I'd like to know the reason of this fact. I heard something about 'external data', and I know that both columns and index also present the 'u' of unicode before the names of these ones. I don't know neither almost anything about encoding and I would be really grateful if someone explains something about this in addition.

I'm using Python 2 and I tryed to solve it column by column with functions as

.astype(str) 
.astype(basestring)
.apply(str)

and

.str.decode('iso-8859-1').str.encode('utf-8')

(I read this last one here and I just wrote it in my code to try another thing). I also tried

unicodedata.normalize('NFKD', df_bolsa[l]).encode('ascii','ignore')

but this last one cannot be used with a series. I hope someone to be able to help me to clarify this matter. Thank you very much in advance!!

How about: http://stackoverflow.com/questions/2365411/python-convert-unicode-to-ascii-without-errors — Stephen Rauch, Feb 23 '17 at 17:21
Thank you but I don't know actually how to apply that problem to mine. Anyway I will read it tomorrow to try to understand something about the encoding... thanks again! — emilio.molina, Feb 23 '17 at 17:32

score 8 · Accepted Answer · edited Apr 27 '20 at 13:59

8

You can use the following code.

for column in df:
    df[column] = df_peru[column].str.encode('utf-8')

edited Apr 27 '20 at 13:59

Jaroslav Bezděk

6,967
6
29
46

answered Apr 04 '17 at 16:44

emilio.molina

327
1
2
12

score 1 · Answer 2 · answered Jul 23 '20 at 10:37

To help others, this version worked for me. I was getting an error while loading my dataframe to an oracle database: "UnicodeDecodeError: 'ascii' codec can't decode byte 0xea in position 2: ordinal not in range(128)"

I am on Python ver 2.7

for column in df:
    df[column]=  df[column].astype(str).str.decode('utf-8')

unicode datas of a dataframe to strings

2 Answers2

Linked