python: are property fields being cached automatically?

Question

My question is are the following two pieces of code run the same by the interpreter:

class A(object):
  def __init__(self):
     self.__x = None

  @property
  def x(self):
     if not self.__x:
        self.__x = ... #some complicated action
     return self.__x

and the much simpler:

class A(object):
  @property
  def x(self):
      return ... #some complicated action

I.e., is the interpreter smart enough to cache the property x?

My assumption is that x does not change - finding it is hard, but once you find it once there is no reason to find it again.

Answer 1

No, the getter will be called every time you access the property.

Answer 2

To anyone who might be reading this in 2020, this functionality is now available in the functools module as part of the standard library as of Python 3.8.

https://docs.python.org/dev/library/functools.html#functools.cached_property

Important to note, classes that define their own __dict__ (or do not define one at all) or use __slots__ might not work as expected. For example, NamedTuple and metaclasses.

Answer 3

No you need to add a memoize decorator:

class memoized(object):
   """Decorator that caches a function's return value each time it is called.
   If called later with the same arguments, the cached value is returned, and
   not re-evaluated.
   """
   def __init__(self, func):
      self.func = func
      self.cache = {}
   def __call__(self, *args):
      try:
         return self.cache[args]
      except KeyError:
         value = self.func(*args)
         self.cache[args] = value
         return value
      except TypeError:
         # uncachable -- for instance, passing a list as an argument.
         # Better to not cache than to blow up entirely.
         return self.func(*args)
   def __repr__(self):
      """Return the function's docstring."""
      return self.func.__doc__
   def __get__(self, obj, objtype):
      """Support instance methods."""
      return functools.partial(self.__call__, obj)

@memoized
def fibonacci(n):
   "Return the nth fibonacci number."
   if n in (0, 1):
      return n
   return fibonacci(n-1) + fibonacci(n-2)

print fibonacci(12)

Answer 4

Properties do not automatically cache their return values. The getter (and setters) are intended to be called each time the property is accessed.

However, Denis Otkidach has written a wonderful cached attribute decorator (published in the Python Cookbook, 2nd edition and also originally on ActiveState under the PSF license) for just this purpose:

class cache(object):    
    '''Computes attribute value and caches it in the instance.
    Python Cookbook (Denis Otkidach) https://stackoverflow.com/users/168352/denis-otkidach
    This decorator allows you to create a property which can be computed once and
    accessed many times. Sort of like memoization.

    '''
    def __init__(self, method, name=None):
        # record the unbound-method and the name
        self.method = method
        self.name = name or method.__name__
        self.__doc__ = method.__doc__
    def __get__(self, inst, cls):
        # self: <__main__.cache object at 0xb781340c>
        # inst: <__main__.Foo object at 0xb781348c>
        # cls: <class '__main__.Foo'>       
        if inst is None:
            # instance attribute accessed on class, return self
            # You get here if you write `Foo.bar`
            return self
        # compute, cache and return the instance's attribute value
        result = self.method(inst)
        # setattr redefines the instance's attribute so this doesn't get called again
        setattr(inst, self.name, result)
        return result

Here is an example demonstrating its use:

def demo_cache():
    class Foo(object):
        @cache
        def bar(self):
            print 'Calculating self.bar'  
            return 42
    foo=Foo()
    print(foo.bar)
    # Calculating self.bar
    # 42
    print(foo.bar)    
    # 42
    foo.bar=1
    print(foo.bar)
    # 1
    print(Foo.bar)
    # __get__ called with inst = None
    # <__main__.cache object at 0xb7709b4c>

    # Deleting `foo.bar` from `foo.__dict__` re-exposes the property defined in `Foo`.
    # Thus, calling `foo.bar` again recalculates the value again.
    del foo.bar
    print(foo.bar)
    # Calculating self.bar
    # 42

demo_cache()

Answer 5

Python 3.2 onwards offers a built-in decorator that you can use to create a LRU cache:

@functools.lru_cache(maxsize=128, typed=False)

Alternatively, if you're using Flask / Werkzeug, there's the @cached_property decorator.

For Django, try from django.utils.functional import cached_property

Answer 6

I've had to look it up, since I had this same question.

The functools package from the standard library will be getting a cached_property decorator as well. Unfortunately, it's only available from Python 3.8 (as of time of this post, it's 3.8a0). The alternative to waiting is to use a custom one, such as this one as mentioned by 0xc0de) or Django's, for now, then switch later:

from django.utils.functional import cached_property
# from functools import cached_property # Only 3.8+ :(

Answer 7

The decorator from Denis Otkidach mentioned in @unutbu's answer was published in O'Reilly's Python Cookbook. Unfortunately O'Reilly doesn't specify any license for code examples – just as informal permission to reuse the code.

If you need a cached property decorator with a liberal license, you can use Ken Seehof's @cached_property from ActiveState code recipes. It's explicitly published under the MIT license.

def cached_property(f):
    """returns a cached property that is calculated by function f"""
    def get(self):
        try:
            return self._property_cache[f]
        except AttributeError:
            self._property_cache = {}
            x = self._property_cache[f] = f(self)
            return x
        except KeyError:
            x = self._property_cache[f] = f(self)
            return x

    return property(get)

Answer 8

Note: Adding for the sake of completeness of available options.

No, property is not cached by default. However there are several options to get that behaviour, I would like to add one more to that:

https://github.com/pydanny/cached-property