0

I have the following python code that is working when I run it using Pycharm :

def get_csv():  # Import data from a csv file
    global df
    import_file_path = filedialog.askopenfilename(filetypes=[("CSV file", '*.csv')])
    if not import_file_path:  # If user press cancel select path display error message
        tk.messagebox.showwarning(title="Error message", message="CSV file import aborted.")
    else:
        # print("CSV path :", import_file_path)
        with open(import_file_path, "rb") as csv_bin:  # automatic CSV encoding detection
            result = chardet.detect(csv_bin.read(10000))
            encode = result['encoding']
            # print("encoding :", encode)
        with open(import_file_path) as csv_file:
            reader = csv.reader(csv_file, delimiter=';')
            first_row = next(reader)
            num_cols = len(first_row)
            if num_cols > 445:
                df = pd.read_csv(import_file_path, sep=';', header=0, na_values=['#NV', ' '], decimal=',', encoding=encode)
                tk.messagebox.showinfo(title="Import success", message="CSV file import successful !")
                return df
            else:
                tk.messagebox.showwarning(title="Error message", message="CSV file not compatible")

I have used pyinstaller to get EXE file corresponding to my python code but I'm getting this error :

Exception in Tkinter callback
Traceback (most recent call last):
  File "tkinter\__init__.py", line 1702, in __call__
  File "import_CSV_python.py", line 30, in get_csv
  File "c:\users\cyrille\appdata\local\programs\python\python37-32\lib\codecs.py
", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 1759: inval
id start byte

I don't have this error when I run my python code using Pycharm. It is working correctly. On my other machine I didn't have any problems with pyinstaller to create working EXE from python file.

I use this command to call pyinstaller and to create the EXE.

pyinstaller --onefile import_CSV_python.py

I have looked online and I found some people that have the same error as I do and their solution is modifying the compat.py file. This sounds complicated for me and I'm completely lost. I have struggled beforehand to have a somewhat working EXE that display the little window with button to launch my different functions. On my other computer (I'm currently working from my parents house from a different machine) running pyinstaller to get working EXE from python file isn't a problem at all it works great.

Edit : I tried to replace the line out = out.decode(encoding) in the compat.py file from pyinstaller package by out = out.decode(encoding, "replace") or out = out.decode(encoding, "ingore") then saving the compat.py file and generating EXE using pyinstaller again with no result. I got the same error : utf8 can't decode

Edit2 : Thanks to John Szakmeister I changed this line

with open(import_file_path, encoding=encode) as csv_file:

It works fine now.

gamer388
  • 53
  • 9
  • have u tried this out? https://stackoverflow.com/questions/47692960/error-when-using-pyinstaller-unicodedecodeerror-utf-8-codec-cant-decode-byt – Delrius Euphoria Aug 04 '20 at 10:32
  • there is nothing much done with compat.py. if thats the only way, u shud go for it. Just find ur python installation directory and go to Lib and then to site-packages and find pyinstaller, open it and edit compat.py and look for `out = out.decode(encoding)` change to `out = out.decode(encoding, errors='ignore')` or `out = out.decode(encoding, 'replace')` – Delrius Euphoria Aug 04 '20 at 10:36
  • Ok I tried replacing out = out.decode(encoding) by out = out.decode(encoding, errors='ignore') or out = out.decode(encoding, 'replace') and saved the compat.py file then regenerate the EXE by running pyinstaller command but I got the same error utf8 can't decode. I don't get why it was working perfectly fine on my another computer but not on the one I'm using right now. The line which is giving the error is first_row = next(reader) – gamer388 Aug 04 '20 at 11:43
  • 1
    Where do you actually use the detected encoding from `chardet`? I would've expected it in the `with open(...) as csv_file` line, but don't see it. I think the line should be `with open(import_file_path, "r", encoding=encode) as csv_file`, so that the correct encoding can be used with `csv.reader()`. Otherwise, open will default to `locale.getpreferredencoding()`, which may not match the file. This could cause the issue you're seeing as `csv.reader()` is lazy and won't read content until you've asked for some data. – John Szakmeister Aug 04 '20 at 12:36
  • I'm using the detected encoding in this line df = pd.read_csv(import_file_path, sep=';', header=0, na_values=['#NV', ' '], decimal=',', encoding=encode) but before in my code I didn't have automatic encoding detection by using chardet (wasn't even using chardet at all) so lines involving chardet wasn't there instead I was using CP1252 encoding hardcoded in the same line involving df = pd.read_csv Ok thanks a lot is it working perfectly – gamer388 Aug 04 '20 at 12:52
  • maybe skip the decoding part, or look for someother way around it, without using it – Delrius Euphoria Aug 04 '20 at 13:44

0 Answers0