Cannot pickle a Python Class instance

Question

Here I have this class definition class definition. When I run below code, it raises following errors.

sm = SaliencyMaskSlic()
operations = [('img_resize', img_resize), ('sal_mask', sm.transform)]
args_list = [{'h_size':258}, {'cropped':True}]

pre_pipeline = Pipeline(ops=operations, arg_list=args_list)
ch = ColorHist('RGB', [6,6,6], [2,2], center=True, pre_pipeline = pre_pipeline)

dill.dump(ch, open('erogol.pkl','wb'))
...
dill.loads('erogol.pkl')


---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-11-c8a5937780b5> in <module>()
----> 1 dill.loads('erogol.pkl')

/usr/local/lib/python2.7/dist-packages/dill/dill.pyc in loads(str)
    158     """unpickle an object from a string"""
    159     file = StringIO(str)
--> 160     return load(file)
    161 
    162 # def dumpzs(obj, protocol=None):

/usr/local/lib/python2.7/dist-packages/dill/dill.pyc in load(file)
    148     pik = Unpickler(file)
    149     pik._main_module = _main_module
--> 150     obj = pik.load()
    151     if type(obj).__module__ == _main_module.__name__: # point obj class to main
    152         try: obj.__class__ == getattr(pik._main_module, type(obj).__name__)

/usr/lib/python2.7/pickle.pyc in load(self)
    856             while 1:
    857                 key = read(1)
--> 858                 dispatch[key](self)
    859         except _Stop, stopinst:
    860             return stopinst.value

/usr/lib/python2.7/pickle.pyc in load_appends(self)
   1185     def load_appends(self):
   1186         stack = self.stack
-> 1187         mark = self.marker()
   1188         list = stack[mark - 1]
   1189         list.extend(stack[mark + 1:])

/usr/lib/python2.7/pickle.pyc in marker(self)
    872         mark = self.mark
    873         k = len(stack)-1
--> 874         while stack[k] is not mark: k = k-1
    875         return k
    876 

IndexError: list index out of range

Basically I have one class instance using another class instance inside. I also used cPickle but it raises as I dump;

TypeError: can't pickle instancemethod objects

Any idea for the solution ?

Possible duplicate: http://stackoverflow.com/questions/6583326/is-this-the-right-way-to-pickle-instance-methods-if-yes-why-isnt-it-in-python and http://stackoverflow.com/questions/5429584/handling-the-classmethod-pickling-issue-with-copy-reg?rq=1 — User, Dec 14 '14 at 20:42
@User: this isn't a duplicate question. With regard to the error, `dill` shouldn't throw an `IndexError` anywhere. The only pickling-related reason I would think might cause that is if you have some dynamically added attribute that is not being saved on pickling. In general, one class instance using another should not be an issue. However, the class definition you link to is large and uses other files in your repository -- and thus makes it less likely for people to etch it and try it out. Any chance you can demonstrate your issue with a smaller code footprint? — Mike McKerns, Dec 15 '14 at 13:18

score 2 · Answer 1 · answered Jan 22 '15 at 14:51

This isn't a pickling error. You can't pickle class instances with pickle or cPickle, but you can with dill. Your code has a bug somewhere that's giving you an IndexError.

Also better than your class having a dump and load method, you might just use dump and load from dill directly... then if you are doing something complicated, you can still add a __getstate__ and __setstate__ method.

Also, your loading from a pickled file, has a bug. You are doing this:

    self = dill.loads(in_path)

While you should (1) be using dill.load instead, and (2) load to _self, and then replace the relevant state.

    _self = dill.load(in_path)
    self.nbins = _self.nbins
    self.mask = _self.mask
    # and so on... (or update all at once using `__dict__`)

Cannot pickle a Python Class instance

1 Answers1