3

Why Python Descriptor Work for Class Level Attribute and not for an Instance Level Attribute.

class PropDescriptor:

    def __init__(self,*args):
        print("Init {} {}".format(type(self),args))
        self.value = 0

    def __get__ (self,instance,owner):
        print("get using descriptor")
        return instance.instance_att

    def __set__(self, instance, value):
        print("set using descriptor")
        instance.instance_att = value


class TestClass:
    class_att = PropDescriptor()

    def __init__(self):
        self.instance_att = PropDescriptor()



t = TestClass()
print("set instance level...")
t.instance_att = 3

print("\nget instance level...")
print(t.instance_att)

print("\nset class level...")
t.class_att = 4

print("\nget class level...")
print(t.class_att)

Output:

Init <class '__main__.PropDescriptor'> ()
Init <class '__main__.PropDescriptor'> ()
set instance level...

get instance level...
3

set class level...
set using descriptor

get class level...
get using descriptor
4

looks like the descriptor is not used for the instance_att.

I found this identical question, but it doesn't really answer the question All answers refer the second question in the post.

The Python documentation specifys:

Instance Binding

If binding to an object instance, a.x is transformed

into the call: type(a).__dict__['x'].__get__(a, type(a)).

But: The first part of the (my code equivalent) statement type(t).__dict__["instance_att"], raises an KeyError as type(t).__dict__ does not have such an attribute. It is an instance level att. Isn't it?

What am I missing?

Community
  • 1
  • 1
OJNSim
  • 736
  • 1
  • 6
  • 22

2 Answers2

4

Its little late for me to reply on this but I'll try my best to explain it. The answer to both parts of your questions lies in the way attribute lookup and setting has been implemented internally for python.

1. Why the descriptor is not being invoked for instance level attributes ?

The answer to this is quite simple once you understand how a.x = somevalue translates internally. When you set an attribute with dot operator like a.x="somevalue" it translates to a.__setattribute__(x, somevalue) which further translates to type(a).__dict__['x'].__set__(x, somevalue).

Now considering your example self.instance_att is defined in the __init__, so it is not present in the type(a).__dict__ which is the dictionary of attributes defined at the class level and hence the descriptor's __set__ never gets called.

2. Why descriptors are only defined at class level ?

I am not hundred percent sure but this is my understanding that because of how __getattribute__ and __setattribute__ are implemented, descriptors will not work at all if defined at instance level (as explained above) and hence they are defined as class attributes only.

Further I believe to make the descriptors work for instance attributes you would have to override __getattribute__ and __setattribute__ for the class. This is how a simplest version __getattribute__ being overridden might look.

def __getattribute__(self, attr):
    attr_val = super().__getattribute__(attr)
    attr_type = type(attr_val)
    if hasattr(attr_type, '__get__'):
        return attr_type.__get__(attr_val, self, type(self))
    return attr_val

Here is the demo for the same

$ python3 -i instance_descriptors.py
>>> t = TestClass()
>>> t.instance_att
get using descriptor
200
>>>

In similar way you need to override __setattribute__ as well to make the attribute setting work.

Hope this helps !

I strongly encourage to go through the below official python documenation for descriptors for complete understanding of the concept.

Descriptors howto

Rohit
  • 3,659
  • 3
  • 35
  • 57
  • I imagine part of the reason that descriptors do not work as instance attributes is due to the descriptor precedence chain: data descriptors > instance attributes > non-data descriptors > `__getattr__`. If an instance attribute could also be a descriptor, it would break this chain, which could *potentially* cause unexpected behavior in several scenarios. One of which would be in this question itself, in which the accessing `t.instance_att` would cause a `RecursionError`. – James Mchugh Dec 02 '19 at 01:56
0

The quick answer: t.instance_att is handled by object.__getattribute__, and its definition simply ignores the value's __get__ and __set__ methods when it is found in the instance dictionary of t.

The longer answer starts with explaining how attribute lookups are handled. They aren't built into the language, but rather handled by the __getattribute__ method of some class. Being rarely overridden, this almost always means object.__getattribute__, which is implemented in C by _PyObject_GenericGetAttrWithDict.

Rather than explain that algorithm in detail here, I refer you to https://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/, which has fantastic diagrams to explain both object-attribute lookup and class-attribute lookup.

There is at least one answer here at Stack Overflow with includes the diagram, but I cannot locate it at the moment, so I apologize for the off-site reference. I'll include a link (or possibly close this as a duplicate) if I can find it.

chepner
  • 497,756
  • 71
  • 530
  • 681
  • What's the idea behind the fact that descriptors only support class level attributes and not instance level? Is there one? – OJNSim May 28 '18 at 06:34
  • I'm going to delete my answer for now, as I can't find a concrete explanation for what right now is just my opinion as to why you get the observed behavior. – chepner May 28 '18 at 14:21
  • I've completely rewritten the answer to emphasize that the observed behavior is simply a consequence of how `object.__getattribute__` is implement, which *reflects* the intended use of descriptors without making it a language-level decision. You could override `__getattribute__` to do whatever you want for your class. – chepner May 28 '18 at 14:50
  • This is a technical explanation for how attribute lookup is conducted.But it does not explain the logic behind. Why class level descriptor is working whereas instance level is handled differently. – OJNSim May 29 '18 at 10:36