1

I'd like to cache intermediate results. I think that a decorator might help.

I not only want to cache based on function arguments, but also based on the implementation.

Background: I want to cache results between program runs because I am experimenting with the individual implementations, and I want to re-use results of the last run if possible.

It seems to be possible to get a function hash with pickle:

from hashlib import sha256
import pickle

def foo(bar):
  print(bar)

h = sha256()
h.update(pickle.dumps(foo))

The next step would now be to use this in a decorator:

from hashlib import sha256
import pickle

def decorated(f):
  def inner(*args, **kwargs):
    h = sha256()
    h.update(pickle.dumps((args, kwargs, f))
    print(h.hexdigest())
    return f(*args, **kwargs)

  return inner

@decorated
def foo(bar):
  print(bar)

foo("bar")

Now pickle complains:

_pickle.PicklingError: Can't pickle <function foo at 0x7f3d177fdee0>: it's not the same object as __main__.foo

I can understand that my original function got re-assigned during the decoration and that this causes trouble (foo=decorared(foo)).

Is pickling the correct choice here at all?

edit

This question here shows how to get the source code of a method: How can I get the source code of a Python function? – maybe that's the better approach?

user4344
  • 193
  • 1
  • 10
  • 1
    No, the problem is you can't pickle nested functions like that. Basically, pickle just uses the name of the function, and then that name is looked up when you unpickle it. – juanpa.arrivillaga Apr 26 '21 at 09:48
  • 1
    Anyway, maybe just the raw bytecode would be sufficient for your purposes? `f.__code__.co_code`? – juanpa.arrivillaga Apr 26 '21 at 09:50
  • That seems to be a solution! – user4344 Apr 26 '21 at 09:52
  • ... with a catch: in `f` when I instantiate an object and change the parameters, this seems not to be reflected in the `co_code`. `inspect.getsource()` contains the `@decorated` which seems ugly, but re-evaluating when I change the decorator name is something I can live with. – user4344 Apr 26 '21 at 11:41

1 Answers1

0

Is pickling the correct choice here at all?

If you want to serialize python function and might use external modules, then you might use dill for that purpose.

Daweo
  • 31,313
  • 3
  • 12
  • 25