Stack Overflow
Stack Overflow

Questions tagged [pickle]

Pickle is an object serialization module for Python. Use this tag together with the Python tag for questions related to storing or loading objects with Pickle.

Pickle provides a powerful set of tools for serializing and de-serializing Python objects.

4936 questions
682
votes
10 answers

How can I use pickle to save a dict (or any other Python object)?

I have looked through the information that the Python documentation for pickle gives, but I'm still a little confused. What would be some sample code that would write a new file and then use pickle to dump a dictionary into it?
Chachmu
  • 7,725
  • 6
  • 30
  • 35
368
votes
10 answers

Python multiprocessing PicklingError: Can't pickle

I am sorry that I can't reproduce the error with a simpler example, and my code is too complicated to post. If I run the program in IPython shell instead of the regular Python, things work out well. I looked up some previous notes on this problem.…
CuriousMind
  • 15,168
  • 20
  • 82
  • 120
364
votes
6 answers

Saving an Object (Data persistence)

I've created an object like this: company1.name = 'banana' company1.value = 40 I would like to save this object. How can I do that?
Peterstone
  • 7,119
  • 14
  • 41
  • 49
340
votes
3 answers

Using pickle.dump - TypeError: must be str, not bytes

I'm using python3.3 and I'm having a cryptic error when trying to pickle a simple dictionary. Here is the code: import os import pickle from pickle import * os.chdir('c:/Python26/progfiles/') def storvars(vdict): f =…
John Rowland
  • 3,579
  • 2
  • 15
  • 7
305
votes
10 answers

Storing Python dictionaries

I'm used to bringing data in and out of Python using CSV files, but there are obvious challenges to this. Are there simple ways to store a dictionary (or sets of dictionaries) in a JSON or pickle file? For example: data = {} data ['key1'] =…
mike
  • 22,931
  • 31
  • 77
  • 100
256
votes
17 answers

Serializing class instance to JSON

I am trying to create a JSON string representation of a class instance and having difficulty. Let's say the class is built like this: class testclass: value1 = "a" value2 = "b" A call to the json.dumps is made like this: t =…
ferhan
  • 2,639
  • 2
  • 14
  • 9
234
votes
14 answers

Can't pickle when using multiprocessing Pool.map()

I'm trying to use multiprocessing's Pool.map() function to divide out work simultaneously. When I use the following code, it works fine: import multiprocessing def f(x): return x*x def go(): pool = multiprocessing.Pool(processes=4) …
ventolin
  • 2,971
  • 3
  • 21
  • 25
225
votes
12 answers

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

I am getting an interesting error while trying to use Unpickler.load(), here is the source code: open(target, 'a').close() scores = {}; with open(target, "rb") as file: unpickler = pickle.Unpickler(file); scores = unpickler.load(); if…
Magix
  • 4,989
  • 7
  • 26
  • 50
208
votes
20 answers

Multiprocessing: How to use Pool.map on a function defined in a class?

When I run something like: from multiprocessing import Pool p = Pool(5) def f(x): return x*x p.map(f, [1,2,3]) it works fine. However, putting this as a function of a class: class calculate(object): def run(self): def f(x): …
Mermoz
  • 14,898
  • 17
  • 60
  • 85
201
votes
7 answers

Saving and loading objects and using pickle

I´m trying to save and load objects using pickle module. First I declare my objects: >>> class Fruits:pass ... >>> banana = Fruits() >>> banana.color = 'yellow' >>> banana.value = 30 After that I open a file called 'Fruits.obj'(previously I…
Peterstone
  • 7,119
  • 14
  • 41
  • 49
182
votes
4 answers

How to read pickle file?

I created some data and stored it several times like this: with open('filename', 'a') as f: pickle.dump(data, f) Every time the size of file increased, but when I open file with open('filename', 'rb') as f: x = pickle.load(f) I can see…
Kenenbek Arzymatov
  • 8,439
  • 19
  • 58
  • 109
182
votes
7 answers

Pickle incompatibility of numpy arrays between Python 2 and 3

I am trying to load the MNIST dataset linked here in Python 3.2 using this program: import pickle import gzip import numpy with gzip.open('mnist.pkl.gz', 'rb') as f: l = list(pickle.load(f)) print(l) Unfortunately, it gives me the…
Neil G
  • 32,138
  • 39
  • 156
  • 257
171
votes
7 answers

best way to preserve numpy arrays on disk

I am looking for a fast way to preserve large numpy arrays. I want to save them to the disk in a binary format, then read them back into memory relatively fastly. cPickle is not fast enough, unfortunately. I found numpy.savez and numpy.load. But…
CuriousMind
  • 15,168
  • 20
  • 82
  • 120
164
votes
8 answers

Pickle or json?

I need to save to disk a little dict object whose keys are of the type str and values are ints and then recover it. Something like this: {'juanjo': 2, 'pedro':99, 'other': 333} What is the best option and why? Serialize it with pickle or with…
Juanjo Conti
  • 28,823
  • 42
  • 111
  • 133
146
votes
1 answer

Dump a list in a pickle file and retrieve it back later

I'm trying to save a list of strings, so that it can be accessed later. How can it be achieved using pickle? An illustrative example could help.
Lewis
  • 1,485
  • 2
  • 11
  • 8
1
2 3
…
99 100