ValueError: unsupported pickle protocol: 4 with pandas

Question

I get this error

ValueError: unsupported pickle protocol: 4

from this line of my code

full_df = pd.read_pickle('df_userID.pickle')

when running the script with python2.7

(on Ubuntu 14.04.5, 3.13.0-95-generic)

Thanks for help.

sure: here is the 1st row of the dataframe https://www.dropbox.com/s/ic7brs7tq5xpt7y/trial_copy.pickle?dl=0 — jjrr, Jan 29 '17 at 11:45
Also check this: http://stackoverflow.com/questions/38466523/python-how-to-handle-the-valueerror-unsupported-pickle-protocol-4-error — Mohammad Yusuf, Jan 29 '17 at 12:41

MaxU - stand with Ukraine · Accepted Answer · 2017-01-29T12:50:42.020

10

It looks like this pickle file has been created like as follows:

pickle.dump(df, file_name, protocol=4)

or

pickle.dump(df, file_name, protocol=-1)

and Python 2.x accepts only protocols: 0, 1, 2

Solution:

either use Pandas pickling or a lower protocol version:

df.to_pickle('/path/to/df.pickle')  # preferred and version independent solution

or:

pickle.dump(df, '/path/to/df.pickle', protocol=2)

another option would be to use HDFStore (H5) or FeatherFormat - both options are very fast and reliable.

edited Jan 29 '17 at 12:50

answered Jan 29 '17 at 12:39

Thanks, now I understand. It was created by one of my collaborator. Another solution is to just use python3, but there I have another issue - that's why I switched to python2. I have problem to update pandas after, I get this error `ImportError: No module named 'pandas.indexes' ` – jjrr Jan 29 '17 at 14:51
@jjrr, you may want to check [this](http://stackoverflow.com/questions/37371451/importerror-no-module-named-pandas-indexes) – MaxU - stand with Ukraine Jan 29 '17 at 14:53
yeah, that's exactly what I did ..but it doesn't work. `pip3 show pandas` gives me still 0.13.1, while `pip show pandas` 0.19.2 – jjrr Jan 29 '17 at 15:04
@jjrr, so consider using one of the following options: HDFStore, Feather or DataFrame.to_pickle() – MaxU - stand with Ukraine Jan 29 '17 at 15:06
- I managed to run the script on my mac...still I would like to solve that issue on the linux cluster, as it's faster ... When I run `pip3.4 install --upgrade pandas` I get a bunch of warnings, like `#warning "Using deprecated NumPy API, disable it by " \` – jjrr Jan 29 '17 at 15:09
i think you should open another question about how to update pandas on Linux... I would recommend to use [Anaconda distributive](https://www.continuum.io/downloads) – MaxU - stand with Ukraine Jan 29 '17 at 15:14
here it is http://stackoverflow.com/questions/41922569/importerror-no-module-named-pandas-indexes-in-ubuntu – jjrr Jan 30 '17 at 09:23

1 Answers1