10

General problem: How can I access a variable in a function's closure?

Specific problem: How can I access the raw cache from a python function wrapped with functools.lru_cache()?

If I memoize a function (example taken from the docs)...

@lru_cache(maxsize=None)
def fib(n):
    if n < 2:
        return n
    return fib(n-1) + fib(n-2)

>>> [fib(n) for n in range(16)]
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610]

>>> fib.cache_info()
CacheInfo(hits=28, misses=16, maxsize=None, currsize=16)

Here is where cache is defined: https://github.com/python/cpython/blob/f0851910eb8e711bf8f22165cb0df33bb27b09d6/Lib/functools.py#L491

When fib() is memoized, lru_cache adds a cache_info() and cache_clear() function to the wrapper. cache_clear() has access to cache and I have access to cache_clear() so can I somehow use that to access cache directly?

gdw2
  • 7,558
  • 4
  • 46
  • 49
  • 1
    It doesn't seem like something you should be able to do, it is a local variable in that function. The only hope you have is reflection. Why are you trying to do this? – Sumner Evans Jul 26 '17 at 21:03
  • 1
    why? because I can. really why? I want to persist cache between reboots while developing. – gdw2 Jul 26 '17 at 21:07
  • 2
    Have you read [this](https://stackoverflow.com/questions/15585493/store-the-cache-to-a-file-functools-lru-cache-in-python-3-2)? – idjaw Jul 26 '17 at 21:10
  • 1
    While you *could* do this, with a lot of fragile poking at implementation details, it'd be better to just use your own memoization implementation. – user2357112 Jul 26 '17 at 21:16
  • related: https://stackoverflow.com/questions/14413946/what-exactly-is-contained-within-a-obj-closure – gdw2 Jul 27 '17 at 15:51

1 Answers1

2

You can use cachier by Shay Palachy. You can tell it to cache to a pickle file.

Alternatively, look at persistent_lru_cache developed by Andrew Barnert.

Sumner Evans
  • 8,951
  • 5
  • 30
  • 47