24

While using:

with open("data_file.pickle", "rb") as pfile:
     raw_data = pickle.load(pfile)

I get the error:

AttributeError: Can't get attribute '_unpickle_block' on <module 'pandas._libs.internals' from '/opt/conda/lib/python3.8/site-packages/pandas/_libs/internals.cpython-38-x86_64-linux-gnu.so'>

Another answer to a similar question suggests checking the version of pickle I am using. It is the same on my machine, where I developed the code and on server, where I am running the code. I have searched everywhere with no answers. Please help.

No-Time-To-Day
  • 361
  • 1
  • 2
  • 10

6 Answers6

21

I don't think the problem is pickle module but Pandas version. Your file was probably created with an older version of Pandas. Now you use a newer version, pickle can't "deserialize" the object because the API change.

Try to downgrade your Pandas version and reload file. You can also try to use pd.read_pickle.

Corralien
  • 109,409
  • 8
  • 28
  • 52
  • I don't know if this is the way to proceed. However, this is what I did: I read data using `pandas v1.4.0` and serialized it as hdf5 file. Then downgraded pandas to `pandas v1.1.5` serialized again using pickle. @Corralien was right, the issue was with pandas and not pickle. – No-Time-To-Day Feb 12 '22 at 10:21
  • 1
    The inverse can also be true. Creating a pickle with a later version, and trying to deserialise with an older version. For example: creating with 1.5.3 and deserialising with 1.2.4. – S3DEV Jul 04 '23 at 14:51
7

In my case I had to upgrade instead of downgrade the Pandas version. Just make sure they match. Some tips for future readers:

Ask the version with:

import pandas as pd
pd.__version__

And change the version with (replace with your own version)

%pip install pandas==1.4.1
Tessa I
  • 108
  • 1
  • 6
  • another tip is to iterate minor version, that is something like `pip install pandas==1.x.0` nad changes the x one by one. Also, now pandas go pandas 2, so you might be checking `pandas==2.x.0` as well. – Muhammad Yasirroni May 07 '23 at 09:30
  • Can also check pandas version with `pip list | grep pandas`. In my case I saved the data using `1.5.2` and could not open with pandas `1.3.0`. – J Agustin Barrachina Aug 29 '23 at 08:02
2

I got this error when using vscode and .net interactive notebooks extension. It got resolved when I updated pandas AND restarted the notebook.

BSalita
  • 8,420
  • 10
  • 51
  • 68
0

Faced same issue. Reading pickle in new version of pandas (the one that it WA originally saved in), saving file as parquet, then reading in older version and saving as pickle helped me.

ADan
  • 41
  • 2
-1

Just change the read function from pickle.load(f) to pd.read_pickle(f) without changing anything in the versions.

# from this :
with open('Data.pkl', 'rb') as f:
     t = pickle.load(f)

# to this:
import pandas as pd 
with open('Data.pkl', 'rb') as f:
      t = pd.read_pickle(f)

This solves the problem for me.

Joooeey
  • 3,394
  • 1
  • 35
  • 49
Mostafa Alheyaly
  • 19
  • 3
-6

all you have to do is downgrade your sklearn version to 1.0.2

doesn't work? make sure the version you had used in your code and env as same.

that'll work.

happy coding!!

Eswar
  • 7
  • 1
  • This does not provide an answer to the question. Once you have sufficient [reputation](https://stackoverflow.com/help/whats-reputation) you will be able to [comment on any post](https://stackoverflow.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/late-answers/32506060) – Rodrigo Rodrigues Aug 23 '22 at 07:10