How to resolve "UnicodeDecodeError" in Pandas 'read_csv' while loading dataset

Question

I am working with a sales dataset that is in the CSV file. When I tried to load this dataset with the pandas read_csv method, I got an error UnicodeDecodeError: 'ascii' codec can't decode byte 0xae in position 16: ordinal not in range(128) I don't know how to solve this issue. I searched for it and got this link, still unable to my problem. I tried the following

import pandas as pd

sales=pd.read_csv("Superstore-Sales.csv")
sales.head(5)

Here is full error

---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_tokens()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_with_dtype()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._string_convert()

pandas/_libs/parsers.pyx in pandas._libs.parsers._string_box_utf8()

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position 16: invalid start byte

During handling of the above exception, another exception occurred:

UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-50-c100e90b0440> in <module>
      1 import pandas as pd
      2 
----> 3 sales=pd.read_csv("Superstore-Sales.csv")
      4 sales.head(5)

D:\Anacoda\lib\site-packages\pandas\io\parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision)
    700                     skip_blank_lines=skip_blank_lines)
    701 
--> 702         return _read(filepath_or_buffer, kwds)
    703 
    704     parser_f.__name__ = name

D:\Anacoda\lib\site-packages\pandas\io\parsers.py in _read(filepath_or_buffer, kwds)
    433 
    434     try:
--> 435         data = parser.read(nrows)
    436     finally:
    437         parser.close()

D:\Anacoda\lib\site-packages\pandas\io\parsers.py in read(self, nrows)
   1137     def read(self, nrows=None):
   1138         nrows = _validate_integer('nrows', nrows)
-> 1139         ret = self._engine.read(nrows)
   1140 
   1141         # May alter columns / col_dict

D:\Anacoda\lib\site-packages\pandas\io\parsers.py in read(self, nrows)
   1993     def read(self, nrows=None):
   1994         try:
-> 1995             data = self._reader.read(nrows)
   1996         except StopIteration:
   1997             if self._first_chunk:

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_column_data()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_tokens()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._convert_with_dtype()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._string_convert()

pandas/_libs/parsers.pyx in pandas._libs.parsers._string_box_utf8()

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position 16: invalid start byte

Please provide a [mcve]. Also, please do not share information as images unless absolutely necessary. See: https://meta.stackoverflow.com/questions/303812/discourage-screenshots-of-code-and-or-errors, https://idownvotedbecau.se/imageofcode, https://idownvotedbecau.se/imageofanexception/. — AMC, Apr 11 '20 at 20:50
Does this answer your question? [utf8' codec can't decode byte 0xae in position](https://stackoverflow.com/questions/26579248/utf8-codec-cant-decode-byte-0xae-in-position) — AMC, Apr 13 '20 at 03:50

score 3 · Accepted Answer · answered Apr 11 '20 at 20:40

3

See if, sales=pd.read_csv("Superstore-Sales.csv", encoding='latin1') helps.

answered Apr 11 '20 at 20:40

tidakdiinginkan

918
9
18

score -1 · Answer 2 · edited Apr 18 '20 at 12:22

-1

You can try this:

address = (r"C:\--------------------/Superstore-Sales.csv")
sales = pd.read_csv(address, encoding='latin1') 
sales.head()

edited Apr 18 '20 at 12:22

Dharman

30,962
25
85
135

answered Apr 18 '20 at 11:43

Digilitiks

1
1

How to resolve "UnicodeDecodeError" in Pandas 'read_csv' while loading dataset

2 Answers2