Pickle , read in data , unsupported pickle protocol: 3 python 2.7

Question

Beginner here, want to read in data with the file ending p.

My code looks like this :

import pickle

training_file = "/home/sk/CarND-Traffic-Sign-Classifier-Project/train.p"
testing_file = "/home/sk/CarND-Traffic-Sign-Classifier-Project/test.p"
with open(training_file, mode='rb') as f:
    train = pickle.load(f)
with open(testing_file, mode='rb') as f:
    test = pickle.load(f)

I get the following error:

ValueError: unsupported pickle protocol: 3

Can someone point out how i can fix it, either changing protocol or reading in the data some other way ?

You get this error by importing pickle ? – iFlo Dec 22 '16 at 13:17 — iFlo, Dec 22 '16 at 13:17
I corrected my post, i get it when i try to load the data – hmmmbob Dec 22 '16 at 13:24 — hmmmbob, Dec 22 '16 at 13:24

score 5 · Answer 1 · answered Dec 22 '16 at 12:57

5

Had the same issue when i created a pickle file using python3 and then tried loading it in python2. Try running your program with python3 or try creating a pickle file using python2.

answered Dec 22 '16 at 12:57

Degraw

513
5
10

1

Is there no other way ? – hmmmbob Dec 22 '16 at 13:16
scomes proposed a good way. More info may be found here [http://stackoverflow.com/questions/28218466/unpickling-a-python-2-object-with-python-3] – Degraw Dec 22 '16 at 14:53

score 3 · Answer 2 · answered Dec 22 '16 at 13:28

3

Pickle uses different protocols to convert your data to a binary stream.

In python 2 there are 3 different protocols (0, 1, 2) and the default is 0. In python 3 there are 5 different protocols (0, 1, 2, 3, 4) and the default is 3. You must specify in python 3 a protocol lower than 3 in order to be able to load the data in python 2. You can specify the protocol parameter when invoking pickle.dump.

It seems as those files was created with a protocol >=3 (probably 3). So the only option you get is to load it into python 3 and then dump it with a lower protocol.

answered Dec 22 '16 at 13:28

iFlo

1,442
10
19

1

Yes.. installed python 3.5 now.. and of course tensorflow is missing and when i try to install it. it says already satisfied in 2.7 :(:( seems like i have to reinstall ubuntu again :( – hmmmbob Dec 22 '16 at 13:30
Reinstalling ubuntu is really not necessary – iFlo Dec 22 '16 at 13:31
You should already or can obtain a pip-3.5. When you launch a pip command, it will take the one who is link to (it's seems its 2.7 in your case) but using /usr/local/bin/pip3.5 and then your command will work. – iFlo Dec 22 '16 at 13:33
I dont see how i can fix it, i installed python 3.5, i can get into it when i enter python3.5 in the terminal. then i tried sudo pip3 install --upgrade $TF_BINARY_URL But still it says no module after i install it sucessfully and try to import it :( – hmmmbob Dec 22 '16 at 13:34

score 1 · Answer 3 · answered Dec 22 '16 at 13:30

Evidently pickle protocol 3 was used in whatever python 3 code pickled the object. You can't unpickle with protocol 3 in python 2. You could however write a short python 3 program that loads it and then dumps it with protocol = 2. Then you can load them in python 2.

https://docs.python.org/2/library/pickle.html#usage

score 0 · Answer 4 · answered Dec 06 '18 at 03:12

0

https://github.com/zopefoundation/zodbpickle

Under Python2, this package forks both Python 2.7's pickle and cPickle modules, adding support for the protocol 3 opcodes.

answered Dec 06 '18 at 03:12

fx-kirin

1,906
1
20
33

Pickle , read in data , unsupported pickle protocol: 3 python 2.7

4 Answers4

Linked