Simulating a 'local static' variable in python

Question

Consider the following code:

def CalcSomething(a):
    if CalcSomething._cache.has_key(a):
      return CalcSomething._cache[a]
    CalcSomething._cache[a] = ReallyCalc(a)
    return CalcSomething._cache[a] 

CalcSomething._cache = { }

This is the easiest way I can think of for simulating a 'local static' variable in python.
What bothers me is that CalcSomething._cache is mentioned outside the function's definition, but the alternative would be something like that:

if not hasattr(CalcSomething, "_cache"):  
    setattr(CalcSomething, "_cache", { } )

inside the function's definition, which is really cumbersome.

Is there a more elegant way?

[EDIT]
Just to clarify, this question is not about local function caches, as the example above might suggest. Here is another short example where a 'static local' might be handy:

def ParseString(s):
    return ParseString._parser.parse(s)  
# Create a Parser object once, which will be used for all parsings.
# Assuming a Parser object is heave on resources, for the sake of this example.
ParseString._parser = Parser()

BTW, you might want to phrase your question differently. You're not looking for a local static variable, but for a way to introduce memoization to your function. — Torsten Marek, Jan 20 '09 at 10:16
your second example is worst than the first one. You could return datetime.datetime.strptime(dts, "%m-%d-%Y %H:%M:%S") since strptime is a class method that creates a new datetime object. There is really no need for a datetime object to be created. — nosklo, Jan 20 '09 at 11:22

score 57 · Accepted Answer · edited Sep 15 '15 at 07:09

57

Turn it into a callable object (since that's what it really is.)

class CalcSomething(object):
    def __init__(self):
        self._cache = {}
    def __call__(self, a):
        if a not in self._cache: 
            self._cache[a] = self.reallyCalc(a)
        return self._cache[a]
    def reallyCalc(self, a):
        return # a real answer
calcSomething = CalcSomething()

Now you can use calcSomething as if it were a function. But it remains tidy and self-contained.

edited Sep 15 '15 at 07:09

smac89

39,374
15
132
179

answered Jan 20 '09 at 10:57

S.Lott

384,516
81
508
779

+1 - another solution that hides caching implementation. This approach didn't cross my mind, but it looks more powerful than my simple decorator. – Abgan Jan 20 '09 at 11:00
For those who want to use custom callables as methods, you'll have to implement __get__ the right way (google for "python descriptor protocol" for more on this) – bruno desthuilliers Jan 20 '09 at 12:33
Beautiful. This combo of using a specialized class plus being callable actually enables a sort of 'functions factory' for different cases, controlled by ctor params. – Paul Oyster Jan 20 '09 at 12:39
1

+1 for __call__, I had no idea you could do this with python objects – Wim Coenen Jan 20 '09 at 12:53
+1: functionality + state -> functor (AKA class with __call__ defined). – cdleary Feb 27 '09 at 04:40

Torsten Marek · Answer 2 · 2009-01-20T14:36:27.477

18

Turn it into a decorator.

def static_var(var_name, initial_value):
    def _set_var(obj):
        setattr(obj, var_name, initial_value)
        return obj
    return _set_var

@static_var("_cache", {})
def CalcSomething(a):
    ...

edited Jan 20 '09 at 14:36

answered Jan 20 '09 at 09:39

Torsten Marek

83,780
21
91
98

1

Do you really consider that 'a more elegant way'? No, seriously ;-) – Paul Oyster Jan 20 '09 at 09:44
I wouldn't consider static variables an elegant pattern to use in Python, but the decorator at least encapsulates the technical details of the attribute setting. – Torsten Marek Jan 20 '09 at 10:05
+1 for cool use of python attributes. @Pau Oyster: no idea why you don't consider this elegant. If you want a non-hackish way to do it, just use a Calculator class with an instance variable. You asked for local static which is inherently ugly. – Wim Coenen Jan 20 '09 at 10:43
Well, being a pragmatist rather than a purist (plus writing C++ code since 1991..) I don't consider local static an ugly thing. – Paul Oyster Jan 20 '09 at 10:56
You need to `return obj` at the end of _set_var. And while I know that you're trying to indicate that _set_var is non-public, there's no need to mangle the name (with the added underscore), as it's only relevant inside static_var. – Jan 20 '09 at 13:29

Abgan · Answer 3 · 2009-01-20T10:16:42.400

Consider writing decorator that will maintain cache and your function won't be contaminated by caching code:

def cacheResults(aFunc):
    '''This decorator funcion binds a map between the tuple of arguments 
       and results computed by aFunc for those arguments'''
    def cachedFunc(*args):
        if not hasattr(aFunc, '_cache'):
            aFunc._cache = {}
        if args in aFunc._cache:
            return aFunc._cache[args]
        newVal = aFunc(*args)
        aFunc._cache[args] = newVal
        return newVal
    return cachedFunc

@cacheResults
def ReallyCalc(a):
    '''This function does only actual computation'''
    return pow(a, 42)

Maybe it doesn't look great at first, but you can use cacheResults() anywhere you don't need keyword parameters. It is possible to create similar decorator that would work also for keyword params, but that didn't seem necessary this time.

Although this is not the aim of my question (see clarification there), this is a beautiful scheme for implementing local caches. Thanks for that. — Paul Oyster, Jan 20 '09 at 10:53
You can get rid of `if not hasattr`... condition by making `_cache` `cacheResults`'s local variable. — GingerPlusPlus, Jul 13 '15 at 17:37

score 4 · Answer 4 · answered Jan 20 '09 at 11:34

One option is to abuse default parameters. ie:

def CalcSomething(a, _cache={}):
    if _cache.has_key(a):

This has the advantage that you don't need to qualify the name, and will get fast local access to the variables rather than doing two slow dict lookups. However it still has the problem that it is mentioned outside the function (in fact it's worse since its now in the function signature.)

To prevent this, a better solution would be to wrap the function in a closure containing your statics:

@apply
def CalcSomething():
    cache = {}  # statics go here

    def CalcSomething(a):
        if cache.has_key(a):
            return cache[a]
        cache[a] = ReallyCalc(a)
        return cache[a]
    return CalcSomething

Unfortunately, apply has been removed from Python 3.0. I too found a handful of similar cases where it was short, simple, and useful. — , Jan 20 '09 at 13:32
I don't think that's a good use of apply. It's much more obvious to just call the function. — Benjamin Peterson, Jan 20 '09 at 15:08

score 4 · Answer 5 · answered Jan 20 '09 at 14:37

4

The solution proposed by S.Lott is the solution I would propose too.

There are useful "memoize" decorators around, too, like:

Given all that, I'm providing an alternative for your initial attempt at a function and a "static local", which is standalone:

def calc_something(a):

    try:
        return calc_something._cache[a]
    except AttributeError: # _cache is not there
        calc_something._cache= {}
    except KeyError: # the result is not there
        pass

    # compute result here

    calc_something._cache[a]= result
    return result

answered Jan 20 '09 at 14:37

tzot

92,761
29
141
204

I guess this will not work. The first pass will run the `except AttributeError` block, initialize cache as empty dict and then fail on keyerror in `calc_something._cache[a]= result` – smido Apr 23 '20 at 09:48
@smido I'm willing to bet that the odds of `dict.__setitem__` throwing a `MemoryError` are much better than it throwing a `KeyError` – tzot Apr 30 '20 at 17:32

Simulating a 'local static' variable in python

5 Answers5

Linked

Related