pickle does not save the code object, just its name. However, giving a function a name is not enough to make it pickleable, because unpickle deserializes it by importing the name. Naturally, this means it must be importable by the unpickling runtime. Functions defed with names can still fail to pickle for this reason. So even if you could convert your lambdas to named functions, that still wouldn't work:
>>> import pickle
>>> def foo():
... def bar():
... pass
... return bar
...
>>> pickle.dumps(foo())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: Can't pickle local object 'foo.<locals>.bar'
You say your reason for not using dill instead is
I've seen a use case where dill can be imported as pickle which is pretty cool but gets confusing in the code since I used pickle and dill for different types of serialization.
But the pickle serialization format is extensible to any internal format you please —json, gzip, even dill. You can have your cake and eat it too!
Let's try it with dill, because that makes it easy:
import pickle
import dill
class DillPickle:
def __init__(self, e):
self.e = e
def __getstate__(self):
return dill.dumps(self.e)
def __setstate__(self, state):
self.e = dill.loads(state)
Pickle normally serializes the state of custom class instances by recursively pickling the object's __dict__. On unpickling, it can then make an uninitialized instance and update its __dict__ with the saved values. If all of its contents are pickleable all the way down, this works. But if not, (as in the case of the lambdas) pickling would fail. But the pickle module provides an interface to override this behavior: __getstate__() for serializing the object's state, and __setstate__() for restoring it from that value. If the return value of __getstate__() is picklable (and __setstate__() is implemented), this works. In this case, we return the picklable bytestring returned by dill.dumps().
Demonstration:
>>> pickle.dumps(DillPickle(lambda x, y: x+y))
b'\x80\x03c__main__\nDillPickle\nq\x00)\x81q\x01C\xe7\x80\x03cdill._dill\n_create_function\nq\x00(cdill._dill\n_load_type\nq\x01X\x08\x00\x00\x00CodeTypeq\x02\x85q\x03Rq\x04(K\x02K\x00K\x02K\x02KCC\x08|\x00|\x01\x17\x00S\x00q\x05N\x85q\x06)X\x01\x00\x00\x00xq\x07X\x01\x00\x00\x00yq\x08\x86q\tX\x1f\x00\x00\x00<ipython-input-18-48b9de2c6f55>q\nX\x08\x00\x00\x00<lambda>q\x0bK\x01C\x00q\x0c))tq\rRq\x0ec__builtin__\n__main__\nh\x0bNN}q\x0fNtq\x10Rq\x11.q\x02b.'
>>> pickle.loads(_).e('foo', 'bar')
'foobar'
Notice that we loaded it with pickle, not dill! And the lambda does not have a name. Of course, dill must be available to the runtime in order to unpickle, but so does the rest of the DillPickle class, just like for any custom type serialized with pickle. dill is now a mere implementation detail used internally by DillPickle. The interface is pickle.
You could even make it a callable, so you don't need to add the .e yourself.
class DillPickleCallable(DillPickle):
def __call__(self, *args, **kwargs):
return self.e(*args, **kwargs)
