Replace property for perfomance gain

Question

Situation

Similar to this question, I want to replace a property. Unlike that question, I do not want to override it in a sub-class. I want to replace it in the init and in the property itself for efficiency, so that it doesn't have to call a function which calculates the value each time the property is called.

I have a class which has a property on it. The constructor may take the value of the property. If it is passed the value, I want to replace the property with the value (not just set the property). This is because the property itself calculates the value, which is an expensive operation. Similarly, I want to replace the property with the value calculated by the property once it has been calculated, so that future calls to the property do not have to re-calculate:

class MyClass(object):
    def __init__(self, someVar=None):
        if someVar is not None: self.someVar = someVar

    @property
    def someVar(self):
        self.someVar = calc_some_var()
        return self.someVar

Problem

The above code does not work because doing self.someVar = does not replace the someVar function. It tries to call the property's setter, which is not defined.

Potential Solution

I know I can achieve the same thing in a slightly different way as follows:

class MyClass(object):
    def __init__(self, someVar=None):
        self._someVar = someVar

    @property
    def someVar(self):
        if self._someVar is None:
            self._someVar = calc_some_var()
        return self._someVar

This will be marginally less efficient as it will have to check for None every time the property is called. The application is performance critical, so this may or may not be good enough.

Question

Is there a way to replace a property on an instance of a class? How much more efficient would it be if I was able to do this (i.e. avoiding a None check and a function call)?

Answer 1

What you are looking for is Denis Otkidach's excellent CachedAttribute:

class CachedAttribute(object):    
    '''Computes attribute value and caches it in the instance.
    From the Python Cookbook (Denis Otkidach)
    This decorator allows you to create a property which can be computed once and
    accessed many times. Sort of like memoization.
    '''
    def __init__(self, method, name=None):
        # record the unbound-method and the name
        self.method = method
        self.name = name or method.__name__
        self.__doc__ = method.__doc__
    def __get__(self, inst, cls):
        # self: <__main__.cache object at 0xb781340c>
        # inst: <__main__.Foo object at 0xb781348c>
        # cls: <class '__main__.Foo'>       
        if inst is None:
            # instance attribute accessed on class, return self
            # You get here if you write `Foo.bar`
            return self
        # compute, cache and return the instance's attribute value
        result = self.method(inst)
        # setattr redefines the instance's attribute so this doesn't get called again
        setattr(inst, self.name, result)
        return result

It can be used like this:

def demo_cache():
    class Foo(object):
        @CachedAttribute
        def bar(self):
            print 'Calculating self.bar'  
            return 42
    foo=Foo()
    print(foo.bar)
    # Calculating self.bar
    # 42

Notice that accessing foo.bar subsequent times does not call the getter function. (Calculating self.bar is not printed.)

    print(foo.bar)    
    # 42
    foo.bar=1
    print(foo.bar)
    # 1

Deleting foo.bar from foo.__dict__ re-exposes the property defined in Foo. Thus, calling foo.bar again recalculates the value again.

    del foo.bar
    print(foo.bar)
    # Calculating self.bar
    # 42

demo_cache()

The decorator was published in the Python Cookbook and can also be found on ActiveState.

This is efficient because although the property exists in the class's __dict__, after computation, an attribute of the same name is created in the instance's __dict__. Python's attribute lookup rules gives precedence to the attribute in the instance's __dict__, so the property in class becomes effectively overridden.

Answer 2

Sure, you can set the attribute in the private dictionary of the class instance, which takes precedence before calling the property function foo (which is in the static dictionary A.__dict__)

class A:
    def __init__(self):
        self._foo = 5
        self.__dict__['foo'] = 10

    @property
    def foo(self):
        return self._foo

assert A().foo == 10

If you want to reset again to work on the property, just del self.__dict__['foo']

Answer 3

class MaskingProperty():
    def __init__(self, fget=None, name=None, doc=None):
        self.fget = fget
        if fget is not None:
            self.name = fget.__name__
        self.__doc__ = doc or fget.__doc__
    def __call__(self, func):
        self.fget = func
        self.name = func.__name__
        if not self.__doc__:
            self.__doc__ = func.__doc__
        return self
    def __get__(self, instance, cls):
        if instance is None:
            return self         
        if self.fget is None:
            raise AttributeError("seriously confused attribute <%s.%s>" % (cls, self.name))
        result = self.fget(instance)
        setattr(instance, self.name, result)
        return result

This is basically the same as Denis Otkidach's CachedAttribute, but slightly more robust in that it allows either:

@MaskingProperty
def spam(self):
    ...

or

@MaskingProperty()     # notice the parens!  ;)
def spam(self):
    ...

Answer 4

You can change what code a function has by replacing the functions's __code__object with the __code__ object from another function.

Here is a decorator function that I created to do just that for you. Feel free to modify it as you see fit. The big thing to remember though is that the both functions need to have the same number of 'free variables' to be swapped like this. This can easily be done by using nonlocal to force it (as shown below).

NULL = object()
def makeProperty(variable = None, default = NULL, defaultVariable = None):
    """Crates a property using the decorated function as the getter.
    The docstring of the decorated function becomes the docstring for the property.

    variable (str) - The name of the variable in 'self' to use for the property
        - If None: uses the name of 'function' prefixed by an underscore

    default (any) - What value to initialize 'variable' in 'self' as if it does not yet exist
        - If NULL: Checks for a kwarg in 'function' that matches 'defaultVariable'

    defaultVariable (str) - The name of a kwarg in 'function' to use for 'default'
        - If None: Uses "default"
        Note: this must be a kwarg, not an arg with a default; this means it must appear after *
    ___________________________________________________________

    Example Use:
        class Test():
            @makeProperty()
            def x(self, value, *, default = 0):
                '''Lorem ipsum'''
                return f"The value is {value}"

        test = Test()
        print(test.x) #The value is 0
        test.x = 1
        print(test.x) #The value is 1

    Equivalent Use:
        @makeProperty(defaultVariable = "someKwarg")
        def x(self, value, *, someKwarg = 0):

    Equivalent Use:
        @makeProperty(default = 0)
        def x(self, value):
    ___________________________________________________________
    """
    def decorator(function):
        _variable = variable or f"_{function.__name__}"

        if (default is not NULL):
            _default = default
        elif (function.__kwdefaults__ is not None):
            _default = function.__kwdefaults__.get(defaultVariable or "default")
        else:
            _default = None

        def fget(self):
            nonlocal fget_runOnce, fget, fset, _default #Both functions must have the same number of 'free variables' to replace __code__
            return getattr(self, _variable)

        def fget_runOnce(self):
            if (not hasattr(self, _variable)):
                fset(self, _default)

            fget_runOnce.__code__ = fget.__code__
            return getattr(self, _variable)

        def fset(self, value):
            setattr(self, _variable, function(self, value))

        def fdel(self):
            delattr(self, _variable)

        return property(fget_runOnce, fset, fdel, function.__doc__)
    return decorator