6

I'm having a hard time understanding what happens when I try to nest descriptors/decorators. I'm using python 2.7.

For example, let's take the following simplified versions of property and classmethod:

class MyProperty(object):
    def __init__(self, fget):
        self.fget = fget
    def __get__(self, obj, objtype=None):
        print 'IN MyProperty.__get__'
        return self.fget(obj)

class MyClassMethod(object):
    def __init__(self, f):
       self.f = f
    def __get__(self, obj, objtype=None):
        print 'IN MyClassMethod.__get__'
        def f(*args, **kwargs):
            return self.f(objtype, *args, **kwargs)
        return f

Trying to nest them:

class A(object):
    # doesn't work:
    @MyProperty
    @MyClassMethod
    def klsproperty(cls):
        return 555
    # works:
    @MyProperty
    def prop(self):
        return 111
    # works:
    @MyClassMethod
    def klsmethod(cls, x):
        return x**2

% print A.klsproperty
IN MyProperty.__get__
...
TypeError: 'MyClassMethod' object is not callable

The __get__ method of the inner descriptor MyClassMethod is not getting called. Failing to figure out why, I tried throwing in (what I think is) a no-op descriptor:

class NoopDescriptor(object):
    def __init__(self, f):
       self.f = f
    def __get__(self, obj, objtype=None):
        print 'IN NoopDescriptor.__get__'
        return self.f.__get__(obj, objtype=objtype)

Trying to use the no-op descriptor/decorator in nesting:

class B(object):
    # works:
    @NoopDescriptor
    @MyProperty
    def prop1(self):
        return 888
    # doesn't work:
    @MyProperty
    @NoopDescriptor
    def prop2(self):
        return 999

% print B().prop1
IN NoopDescriptor.__get__
IN MyProperty.__get__
888
% print B().prop2
IN MyProperty.__get__
...
TypeError: 'NoopDescriptor' object is not callable

I don't understand why B().prop1 works and B().prop2 does not.

Questions:

  1. What am I doing wrong? Why am I getting a object is not callable error?
  2. What's the right way? e.g. what is the best way to define MyClassProperty while re-using MyClassMethod and MyProperty (or classmethod and property)
shx2
  • 61,779
  • 13
  • 130
  • 153
  • 1
    The key to understanding this intuitively is that `@MyProperty` creates something that works like a _data_ attribute, not a method, so you can't nest it with a method descriptor. That intuitive understanding only gets you so far; the full story is in isedev's answer. – abarnert Mar 22 '13 at 23:41

3 Answers3

6

In this case, when the decorators are used without parameters, a decorator is called with the function it decorates as its parameter. The decorator's return value is used instead of the decorated function. So:

@MyProperty
def prop(self):
    ...

is equivalent to:

def prop(self):
    ...
prop = MyProperty(prop)

Since MyProperty implements the descriptor protocol, accessing A.prop will actually call A.prop.__get__(), and you've defined __get__ to call the object which was decorated (in this case, the original function/method), so everything works fine.

Now, in the nested case:

@MyProperty
@MyClassMethod
def prop(self):
    ...

The equivalent is:

def prop(self):
    ...
prop = MyClassMethod(prop)   # prop is now instance of MyClassMethod
prop = MyProperty(prop)      # prop is now instance of MyProperty
                             # (with fget == MyClassMethod instance)

Now, as before, accessing A.prop will actually call A.prop.__get__() (in MyProperty) which then tries to call the instance of MyClassMethod (the object which was decorated and stored in the fget attribute).

But the MyClassMethod does not have a __call__ method defined, so you get the error MyClassMethod is not callable.

And to address your second question: A property is already a class attribute - in your example, accessing A.prop will return the value of the property in the class object and A().prop will return the value of the property in an instance object (which can be the same as the class object if the instance did not override it).

isedev
  • 18,848
  • 3
  • 60
  • 59
  • Your explanation if clear. Thank you. Regarding the second answer: I'm not sure I understand what you mean. I'm looking for something parallel to `classmethod`, meaning I can call it as either `A.prop` or `A().prop`, and in both cases the underlying function is passed a `cls` arg. – shx2 Mar 23 '13 at 14:00
  • In that case, you can check the type of the `obj` argument: if it's a class instance, you can extract the class from it and pass that to the decorated function instead of the instance object. – isedev Mar 23 '13 at 14:06
  • You're missing the point... My question isn't about how to implement a `classproperty` decorator. It's about nesting descriptors. The logic you suggest is already implemented in `classmethod`. I would expect there should be a clean way to reuse/combine `property` and `classmethod` to achieve that, instead of re-implementing the logic in them. But I'm starting to think there isn't... – shx2 Mar 23 '13 at 14:47
  • But they DO NOT apply to the same thing: `property` applies to descriptors and `classmethod` applies to a callable (method) - two different protocols. How do you expect to combine them? – isedev Mar 23 '13 at 14:55
  • So the part I probably don't understand is why "property applies to descriptors". You can do `foo = property(foo)` where `foo` acts as the `fget`, and `foo` is callable and will get called when you access `self.foo`. I thought this means "property applies to callables", much like `classmethod`. – shx2 Mar 23 '13 at 15:17
  • Think of it like this: you access a property as a variable "A.prop = value" but you call a method "A.method()". Even though the descriptor is callable, you don't pass arbitrary arguments to it (just the object) but you pass arbitrary arguments to a method (depends on the method signature). – isedev Mar 23 '13 at 15:22
4

You can make your code work if you make MyProperty apply the descriptor protocol to its wrapped object:

class MyProperty(object):
    def __init__(self, fget):
        self.fget = fget
    def __get__(self, obj, objtype=None):
        print('IN MyProperty.__get__')
        try:
            return self.fget.__get__(obj, objtype)()
        except AttributeError: # self.fget has no __get__ method
            return self.fget(obj)

Now your example code works:

class A(object):
    @MyProperty
    @MyClassMethod
    def klsproperty(cls):
        return 555

print(A.klsproperty)

The output is:

IN MyProperty.__get__
IN MyClassMethod.__get__
555
Blckknght
  • 100,903
  • 11
  • 120
  • 169
  • Thanks, this makes sense. But is this the recommended approach? If so, how come the builtin `property` and `classmethod` don't behave this way (and can't be nested)? – shx2 Mar 23 '13 at 13:57
  • I'm not sure if this is discouraged, or why the builtin `property` doesn't do this already. I do know that there are some limitations that this does not fix though. For one thing, you can't nest the descriptors in the reverse order, even if you made `MyClassMethod` more clever. And I don't think the `__set__` descriptor function is called for class variable assignment, so you need to do some metaclass magic for writable classproperties to work. Perhaps somebody with more descriptor programming experience can weigh in on other issues and limitations. – Blckknght Mar 23 '13 at 22:15
0

I found the definitive answer to my own old question in Graham Dumpleton's fascinating blog.

In short, the decorators I wrote do not honour the descriptors protocol, by trying to call the wrapped function/object directly, instead of first giving them a chance to perform their "descriptor magic" (by calling their __get__() first).

shx2
  • 61,779
  • 13
  • 130
  • 153