Callable object decorator applied to method doesn't get self argument on input

Question

import functools


class Decor(object):
    def __init__(self, func):
         self.func = func

    def __call__(self, *args, **kwargs):
        def closure(*args, **kwargs):
            print args, kwargs
            return self.func(*args, **kwargs)
        return closure(*args, **kwargs)


class Victim(object):
    @Decor
    def sum(self, a, b):
        return a+b


v = Victim()
v.sum(1, 2)

Results in:

(1, 2) {}
Traceback (most recent call last):
  File "test.py", line 19, in <module>
    v.sum(1, 2)
  File "test.py", line 11, in __call__
    return closure(*args, **kwargs)
  File "test.py", line 10, in closure
    return self.func(*args, **kwargs)
TypeError: sum() takes exactly 3 arguments (2 given)

How do I get self argument for the method?

UPDATE: I've managed to create a more useful adaptation of Martijn's answer, which returns Decor object in response to __get__, but at the same time binds self argument, when it is called as a method of object. With this version you can say e.g. Victim.sum.hooks.append(my_favorite_function) and my_favorite_function will be called before Victim.sum. WARNING: this version is thread-unsafe.

class Decor(object):
    def __init__(self, func):
        self.func = func
        self.hooks = []
        wraps(self.func)(self)

    def __get__(self, instance, klass):
        if instance != None: self.instance = instance
        if klass != None: self.klass = klass
        return self

    def __call__(self, *args, **kwargs):
        def closure(*args, **kwargs):
           for function in self.hooks:
               function(*args, **kwargs)
           func = self.func
           retval = func(*args, **kwargs) #kwargs_copy #called with notify = False
           return retval
        return closure.__get__(self.instance, self.klass)(*args, **kwargs)

Easy way: Have the decorator return a function instead of a class instance. Less easy way: Implement the [descriptor protocol](http://docs.python.org/2/howto/descriptor.html), just like a real function. — user2357112, Mar 20 '14 at 21:48
You named your nested function `__closure__` but you then call `closure()`. Is that a typo? — Martijn Pieters, Mar 21 '14 at 10:18
Oh, sorry, it's a typo. I fixed it in console before while testing, but forgot to fix it in question. — Boris Burkov, Mar 21 '14 at 10:23

Martijn Pieters · Accepted Answer · 2014-03-22T16:17:35.477

7

Python functions act as descriptors, which means that whenever you access a function on a class or instance, their .__get__() method is invoked and a method object is returned which keeps a reference to the original function, and for instances, a reference to the instance. Method object then acts as wrappers; when called they call the underlying function and pass in the instance reference as self.

Your callable class object, on the other hand, does not implement the descriptor protocol, it has no .__get__() method, and thus it never is given an opportunity to bind to the instance. You'll have to implement this functionality yourself:

class Decor(object):
    def __init__(self, func):
         self.func = func

    def __get__(self, instance, owner):
        if instance is None:
            return self
        d = self
        # use a lambda to produce a bound method
        mfactory = lambda self, *args, **kw: d(self, *args, **kw)
        mfactory.__name__ = self.func.__name__
        return mfactory.__get__(instance, owner)

    def __call__(self, instance, *args, **kwargs):
        def closure(*args, **kwargs):
            print instance, args, kwargs
            return self.func(instance, *args, **kwargs)
        return closure(*args, **kwargs)

Demo:

>>> class Victim(object):
...     @Decor
...     def sum(self, a, b):
...         return a+b
... 
>>> v = Victim()
>>> v.sum
<bound method Victim.sum of <__main__.Victim object at 0x11013d850>>
>>> v.sum(1, 2)
<__main__.Victim object at 0x11013d850> (1, 2) {}
3

It is not a good idea to store the instance you are bound to directly on the Decor instance; this is a class attribute, shared among instances. Setting self.instance is neither thread-safe nor allows methods to be stored for later invocation; the most recent __get__ call will alter self.instance and lead to hard-to-resolve bugs.

You can always return a custom proxy object instead of a method:

class DecorMethod(object):
    def __init__(self, decor, instance):
        self.decor = decor
        self.instance = instance

    def __call__(self, *args, **kw):
        return self.decor(instance, *args, **kw)

    def __getattr__(self, name):
        return getattr(self.decor, name)

    def __repr__(self):
        return '<bound method {} of {}>'.format(self.decor, type(self))

and use that in your Decor.__get__ instead of producing a method:

def __get__(self, instance, owner):
    if instance is None:
        return self
    return DecorMethod(self, instance)

The DecorMethod here passes any requests for unknown attributes back to the Decor decorator instance:

>>> class Victim(object):
...     @Decor
...     def sum(self, a, b):
...         return a + b
... 
>>> v = Victim()
>>> v.sum
<bound method <__main__.Decor object at 0x102295390> of <class '__main__.DecorMethod'>>
>>> v.sum.func
<function sum at 0x102291848>

edited Mar 22 '14 at 16:17

answered Mar 21 '14 at 10:19

Martijn Pieters

1,048,767
296
4,058
3,343

Thanks for a clear explanation. I've stumbled upon another example in Python Decorator Library, before finding out that you posted your answer: https://wiki.python.org/moin/PythonDecoratorLibrary#Class_method_decorator_using_instance – Boris Burkov Mar 21 '14 at 10:44
Martijn, could you suggest a way to adapt your solution for the following goal? I'd like to store a list of hook functions in `Decor` and have `Decor.__call__` call each of them before calling the decorated function (e.g. `Victim.sum`). I'll need to update the list of hooks in runtime by saying `Victim.sum.hooks.append(another_func)`. At the same time I'll need to be able to bind `self` argument to the decorated function (e.g. `Victim.sum` should get `Victim` instance as `self` argument). Is it possible to achieve both of these capacities at the same time? – Boris Burkov Mar 21 '14 at 21:01
@Bob: That's fairly easy; I've made the `instance` object being passed into the `__call__` method of your `Decor` object more explicit; you can add any number of extra functions in there, those don't *have* to have `instance` passed into them, but you do need to pass it into the decorated function (`self.func`). – Martijn Pieters Mar 21 '14 at 21:36
Thank you for answering, but the way you suggested, you can't say `v.sum.hooks.append(another_function)`: `v.sum` is a bound method, generated from `mfactory`, not `Decor` callable object. It doesn't allow to access Decor's attributes. In fact, when I was making `Decor` a class, not a function, my purpose was to store the hooks in its attributes. I wanted to have `__get__` return the `Decor` instance itself and seems, that my adaptation of your answer (updated question with it) works. Thanks! – Boris Burkov Mar 22 '14 at 14:34
You *can* access the decorator on the class (the `__get__` methods special-cases this). – Martijn Pieters Mar 22 '14 at 15:44
@Bob: your approach has some serious flaws; you are modifying the decorator instance directly, which means that a) you made the code thread-unsafe, and b) if you stored two such 'methods' in references the most recent binding wins. This will lead to some very strange errors. – Martijn Pieters Mar 22 '14 at 15:48
@Bob: My approach re-used the existing `MethodType` proxy object (via a `lambda` shortcut), you can also return your own custom proxy object that fills the same role, *and* lets you add additional functionality. Because you create separate instances to model the binding (like a method would) you keep this thread-safe and safe to store references for later invocation. – Martijn Pieters Mar 22 '14 at 15:50
@Bob: I've expanded the answer to show you how to write a custom method proxy instead. – Martijn Pieters Mar 22 '14 at 16:21
Indeed, my version is thread unsafe. With Proxy approach, you can't say `v.sum.hooks.append(function)`, but you can access Decorator object from `class`, not from `object`. Thanks for you help, Martijn. I have a feeling, that tinkering with overly sophisticated decorators can make my project fragile and unfriendly for users. I'll at least avoid storing hooks in decorator itself and move them to instance attributes. – Boris Burkov Mar 24 '14 at 12:59
@Bob: with the `DecorMethod` you **can** use `v.sum.hooks.append()` as the `hooks` attribute is proxied. – Martijn Pieters Mar 24 '14 at 13:10
Ah, __getattr__ serves that purpose. Got it. Cheers! – Boris Burkov Mar 24 '14 at 18:03

Callable object decorator applied to method doesn't get self argument on input

1 Answers1

Linked

Related