How to resolve pickle encoding?

Question

I am facing an issue with saving my pickle files. I receive the error when I open the pickle file manually:

Error: C:\Users\df.pkl is not UTF-8 encoded. Saving disabled. See console for more details.

This is what I did for the error to occur:

pickle.dump(df, open('df.pkl', 'rb'))

and then subsequently:

df = pickle.load(open('df.pkl', 'rb'))

Encoding shouldn't matter when writing to a binary file. I think you need to open the file with mode `'wb'` to output to it. See my answer to [Saving an Object (Data persistence)](https://stackoverflow.com/questions/4529815/saving-an-object-data-persistence) for examples of using `pickle`. — martineau, Jan 20 '20 at 01:48
You should be opening the pickle dump file in write mode 'wb' — spacecowboy, Jan 20 '20 at 01:51
I strongly recommend using context managers to handle those file objects! — AMC, Jan 20 '20 at 02:19
I changed the pickle dump file to write mode 'wb' but error persists. I am pickling a dataframe btw, does that affect? — Shauna Loh, Jan 20 '20 at 03:13

Renaud · Answer 1 · 2020-07-17T19:52:48.980

I'm not sure if your issue is not coming from encoding of your script: try by adding at the beginning of your code:

#coding: utf-8

Maybe another solution can be to use the to_pickle function of pandas (As I think you want to pickle a DataFrame): https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_pickle.html

To save:

df.to_pickle("C:\Users\df.pkl")

To load:

df = pd.read_pickle("C:\Users\df.pkl")

1 Answers1