1

According to the documentation, this code seems like it should pickle various elements of a function and allow it to be unpickled, even if it did not exist in the current namespace. However, no change appears.

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> def average(*args):
...     return sum(args) / len(args)
...
>>> import pickle
>>> ap = pickle.dumps(average)
>>> del average
>>> average = pickle.loads(ap)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'average'
>>> ap
b'\x80\x03c__main__\naverage\nq\x00.'
>>>

As you can see, it is impossible to unpickle a function that does not exist. Security issues aside, the following code was expected to fix the problem and allow functions to be pickled and unpickled.

>>> import marshal, types, copyreg
>>> copyreg.pickle(types.CodeType,
...                lambda code: (marshal.loads, (marshal.dumps(code),)),
...                marshal.loads)
...
>>> up = lambda co, ns, de, cl: types.FunctionType(co, globals(), na, de, cl)
>>> copyreg.pickle(types.FunctionType,
...                lambda function: (up, (function.__code__,
...                                       function.__name__,
...                                       function.__defaults__,
...                                       function.__closure__)),
...                up)
...
>>> def average(*args):
...     return sum(args) / len(args)
...
>>> pickle.dumps(average)
b'\x80\x03c__main__\naverage\nq\x00.'
>>>

One would expect that if the code were working properly, the bytes emitted by dumping the function would be different than before. The same code was generated, and unpickling should be similar.

>>> del average; average = pickle.loads(_)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'average'
>>>

It was easy enough to confirm that theory. Therefore, the question still remains on how to pickle a function natively with the API instead of wrapping it another layer. That is an easy but sloppy solution.

Question: How can you use the pickle API to have it capture a function and not just its name? How can the default handling for serializing functions be overridden, capturing value instead of reference?

Reference: Question 1253528 was interesting but does not alter how functions are natively pickled.

Community
  • 1
  • 1
Noctis Skytower
  • 21,433
  • 16
  • 79
  • 117
  • 5
    Your opening statement doesn't mesh with what I take from the docs: *"Note that functions (built-in and user-defined) are pickled by “fully qualified” name reference, not by value. This means that only the function name is pickled, along with the name of the module the function is defined in. Neither the function’s code, nor any of its function attributes are pickled. Thus the defining module must be importable in the unpickling environment..."* - from [your own link](http://docs.python.org/3.3/library/pickle.html#what-can-be-pickled-and-unpickled). – Gareth Latty Jan 14 '13 at 21:04
  • Thank you! The question has been edited to be clearer. Yes, it is obvious how functions are pickled after looking at the byte code created. My goal was to capture most of the function's value instead of its reference. – Noctis Skytower Jan 14 '13 at 21:11
  • 1
    @Lattyware: maybe write that as an answer for future reference of those who read this. The op is mistaken, but I think it's a topic misunderstood enough to warrant keeping the question. – Logan Jan 14 '13 at 21:11
  • @Mark: As I mentioned in the reference, the answer to this question in not presented there. It should be possible to override the default handling for pickling functions instead of creating a workaround. – Noctis Skytower Jan 15 '13 at 04:38

0 Answers0