'utf-8' codec can't decode byte 0x92 in position 11: invalid start byte

Question

I am trying to convert all csv files of a folder to xlsx and using below code.

import glob
import csv
import pandas as pd
files = glob.glob('D:\cf111\*.csv')
for k in files:
    df = pd.read_csv(k)
    df.to_excel("abc.xlsx")

The code is generating the below error.

utf-8' codec can't decode byte 0x92 in position 11: invalid start byte

I cant sort out, how to resolve this error.

use raw string when you have backslashes `r'D:\cf111\*.csv'` — Chris Adams, Mar 10 '20 at 08:29
Please post the full traceback of the exception you are getting. — user4815162342, Mar 10 '20 at 08:29
This means the file is *not* UTF8. Perhaps it's Latin1 with a fancy quote, like the duplicate? — Panagiotis Kanavos, Mar 10 '20 at 08:31
Does this answer your question? ['utf-8' codec can't decode byte 0x92 in position 18: invalid start byte](https://stackoverflow.com/questions/46000191/utf-8-codec-cant-decode-byte-0x92-in-position-18-invalid-start-byte) — Panagiotis Kanavos, Mar 10 '20 at 08:31

score 1 · Answer 1 · answered Mar 10 '20 at 09:08

1

The problem might because there are contents in the csv that do not support encoding="utf-8". you can try using other encoding though.

code example:

import glob
import csv
import pandas as pd
files = glob.glob('D:\cf111\*.csv')
for k in files:
    df = pd.read_csv(k, encoding='ISO 8859-1')
    df.to_excel("abc.xlsx")

document reference: https://docs.python.org/3/library/codecs.html#standard-encodings

answered Mar 10 '20 at 09:08

Lurima

52
3

Thank you. The encoding problem is resolved. but it only converts one(first) file. Does not run through the whole folder to check other files. – Husnain Iqbal Mar 11 '20 at 04:01
You need to change your excel file names each time the **for loop** executes the loop. Else, only your **last** csv will be converted. – Lurima Mar 11 '20 at 05:08

score 1 · Answer 2 · edited May 18 '23 at 21:54

1

Use the ISO 8859-1 encoding instead of 'utf-8', for example

msg = email.message_from_string(str(arr[1],  'ISO 8859-1'))

edited May 18 '23 at 21:54

A. K.

34,395
15
52
89

answered Mar 02 '21 at 18:21

Ray

124
7

'utf-8' codec can't decode byte 0x92 in position 11: invalid start byte

2 Answers2