Is it possible to decorate a Python class whose instances will be pickled?

Question

The following code run correctly:

import pickle

class MyClass():   
    def __init__(self, arg):
        self.arg = arg

a = MyClass('my arg')
with open('/home/mahikeulbody/mypickle', 'wb') as file:
    pickle.dump(a, file)

but adding a decorator to get a multiton class :

import pickle

def multiton(cls):
    instances = {}
    def getinstance(arg):
        if arg not in instances:
            instances[arg] = cls(arg)
        return instances[arg]
    return getinstance

@multiton
class MyClass():   
    def __init__(self, arg):
        self.arg = arg

a = MyClass('my arg')
with open('/home/michel/mypickle', 'wb') as file:
    pickle.dump(a, file)

produces the following error:

pickle.dump(a, file)
_pickle.PicklingError: Can't pickle <class '__main__.MyClass'>: it's not the same object as __main__.MyClass

What is wrong ?

Uh, this decorator has more problems than just that. It's just not a class, it has become a factory function. For example, you can't type checks with it or subclass it. — , Jan 24 '13 at 17:25

score 2 · Accepted Answer · answered Jan 24 '13 at 17:26

2

Pickle must be able to load the class directly. Your decorator replaces the class with a factory function, making it impossible for pickle to import the class itself.

Use a separate factory function, not a decorator, returning a 'private' class (but still importable directly):

class _MyClass():   
    def __init__(self, arg):
        self.arg = arg

def MyClass(arg, instances={}):
    if arg not in instances:
        instances[arg] = _MyClass(arg)
    return instances[arg]

answered Jan 24 '13 at 17:26

Martijn Pieters

1,048,767
296
4,058
3,343

A side effect of your answer is that I discovered with surprise that instances variable is persistent among different call of the function. This shocks me a little bit... – mahikeulbody Jan 24 '13 at 17:55
@user2008329: See ["Least Astonishment" in Python: The Mutable Default Argument](http://stackoverflow.com/q/1132941). It's perfectly reasonable when you realize how functions work. :-) – Martijn Pieters Jan 24 '13 at 17:57

Mike McKerns · Answer 2 · 2013-10-17T14:17:56.247

To do this, I'd use dill, which can serialize almost anything in python.

>>> def multiton(cls):
...     instances = {}
...     def getinstance(arg):
...         if arg not in instances:
...             instances[arg] = cls(arg)
...         return instances[arg]
...     return getinstance
... 
>>> @multiton
... class MyClass():   
...     def __init__(self, arg):
...         self.arg = arg
... 
>>> import dill
>>>                       
>>> a = MyClass('my arg')
>>> b = dill.loads(dill.dumps(a))
>>> a
<__main__.MyClass instance at 0x4d64558>
>>> b
<__main__.MyClass instance at 0x4d64800>

Dill also has some good tools for helping you understand what is causing your pickling to fail when your code fails.

Is it possible to decorate a Python class whose instances will be pickled?

2 Answers2