unpickling '_io.BufferedReader' error

Question

Hello everyone I pickled a list of dictionaries to a file using the below code

fd = open(file_name,'ab')
for i in listOFDicts:
    pickle.dump(i,fd)
fd.close()

Then Now I want to load it using the below code

with open(filename, 'rb') as  pickleFile:
    content = pickle.loads(pickleFile)

and I get this error "a bytes-like object is required, not '_io.BufferedReader'"

when I use load() instead of loads()

I get this error

"UnicodeDecodeError: 'ascii' codec can't decode byte 0xc6 in position 0: ordinal not in range(128)"

and if I used pickle.load(filename, encoding='latin1')

I get this error "ModuleNotFoundError: No module named 'bson'"

any help would be really appreciated

Sincerely

that should work. Are you sure you're not using `loads` instead of `load` ? full traceback please — Jean-François Fabre, Mar 29 '18 at 13:09
you _have_ to use `load`. Why not using `json` to store your dictionaries? using append mode could leave data in the file. Use `wb` to create the file. — Jean-François Fabre, Mar 29 '18 at 13:24
dump a list of dict, don't loop on the dicts. and appending isn't a very good idea. You don't know what was before. — Jean-François Fabre, Mar 29 '18 at 13:42
it might be not a good idea to append ... but its needed .. or you suggest to replace appending by multiple files ? ... and also what is the difference between dumping a list of dicts and looping over the dicts? ... I mean what unicode difference that will make? — Omar K. Aly, Mar 29 '18 at 13:46
I'd store as a list of dicts. After that, I'd read back the previous list of dicts, append the new dicts to the list and pickle the whole thing... or I'd use `json` so the serialized data is readable (pickle is binary) — Jean-François Fabre, Mar 29 '18 at 13:47

HamishLacmane · Answer 1 · 2018-03-29T13:39:24.967

0

I wrote the following code up to show you how to use pickle, edit it to your needs.

import pickle

dict = {0:0, 1:1}

list_of_dict = [dict, dict]

with open("save.pickle", "wb") as f:
    pickle.dump(list_of_dict, f)

del list_of_dict
#print(list_of_dict)
#put below in the other script.
with open("save.pickle", "rb") as f:
    list_of_dict = pickle.load(f)
print(list_of_dict)

This was written in python 3, if you are using python 2, change the following lines:

import pickle

to

import cPickle as pickle

and change the prints appropriately.

The del command was used to delete the variable to save me time from having to write it in 2 scripts. You may ignore that line and then write the remainder in the second script.

Source: Several of my own python scripts used in past and the following: https://wiki.python.org/moin/UsingPickle

EDIT: I've improved the code. Try and use it similarly.

edited Mar 29 '18 at 13:39

answered Mar 29 '18 at 13:30

HamishLacmane

88
1
11

Thanks for your answer. the idea is that I need to append to the file which I did and then I got the errors I mentioned in my questions it would be appreciated if you could help me go through solving those errors .... Sincerely – Omar K. Aly Mar 29 '18 at 13:34
In that case you need to load the pickled data first, append to the list, and then pickle dump it again. Have a look at the answer here to append to existing data: https://stackoverflow.com/questions/28077573/python-appending-to-a-pickled-list – HamishLacmane Mar 29 '18 at 13:41
@Jean-FrançoisFabre it works, however cPickle is faster. Have a read of the documentation or wiki page I linked in my answer :) – HamishLacmane Mar 29 '18 at 13:43

unpickling '_io.BufferedReader' error

1 Answers1