1

I have the following code where I am trying to download a csv

fs = gcsfs.GCSFileSystem(project='<project ID>', token='<path to service account json>')
    
with fs.open('gs://some_path_to_csv') as f:
            stability_data = pd.read_csv(f, delimiter=',', encoding='UTF-16 LE')
            return stability_data

This code works perfectly on Mac, but doesn't work on Windows. On Windows I am getting the following data:

    ÿþD  Unnamed: 1  Unnamed: 2  Unnamed: 3
0  NaN         NaN         NaN         NaN
1  NaN         NaN         NaN         NaN
2  NaN         NaN         NaN         NaN
   ÿþD  Unnamed: 1  Unnamed: 2  Unnamed: 3
0  NaN         NaN         NaN         NaN
1  NaN         NaN         NaN         NaN
2  NaN         NaN         NaN         NaN

Looks like an issue with encoding or conversion from bytes to string. But I still haven't figured out what the issue is. Please help. Thanks!

UPDATE: The issue seems to be with Python 3.x and not Windows/Mac. If I use Python 2.7 the code runs, but if I use Python 3.7 it fails with the below error:

UnicodeDecodeError: 'utf-16-le' codec can't decode byte 0x74 in position 0: truncated data

xertzer
  • 69
  • 6
  • maybe you need different value in `encoding=...` - As I know Mac can use different UNICODE to display UTF. – furas Apr 01 '21 at 12:03
  • I don't know if it will help but it shows that Mac and Windows may use UNICODE in different way: [Unicode encoding for filesystem in Mac OS X not correct in Python?](https://stackoverflow.com/questions/9757843/unicode-encoding-for-filesystem-in-mac-os-x-not-correct-in-python) – furas Apr 01 '21 at 12:06
  • The issue seems to be with python version. It works for 2.7 but doesn't work for 3.7. – xertzer Apr 01 '21 at 14:32
  • always put full error message (starting at word "Traceback") in question (not comment) as text (not screenshot, not link to external portal). There are other useful information. – furas Apr 01 '21 at 14:52

0 Answers0