14

I want to know that if I have class like this:

class TestClass(object):
    def __init__(self):
        self.a = 20
        self.b = 30
    
obj = TestClass()

When I write obj.a, which of the following is called first?

  • TestClass.__getattribute__("a")
  • TestClass.__getattr__("a")
  • getattr(obj, "a")
  • obj.__dict__['a']

I have a similar question for setattr

According to Python 2.7 docs:

object._getattr_(self, name)

Called when an attribute lookup has not found the attribute in the usual places (i.e. it is not an instance attribute nor is it found in the class tree for self). name is the attribute name. This method should return the (computed) attribute value or raise an AttributeError exception.

It says "not found [...] in usual places". What are the "usual places". I want to know when __getattr__ is called.

Also, what is the difference between __getattr__ and __getattribute__?

Can anyone give me example where all of these are used?

Toothpick Anemone
  • 4,290
  • 2
  • 20
  • 42
user196264097
  • 877
  • 2
  • 13
  • 23
  • I can't understand the question. – LtWorf Feb 09 '13 at 10:52
  • None of your 3 options exist. Did you mean `__getattribute__('a')`, `__getattr__('a')`, and `__dict__['a']` – Eric Feb 09 '13 at 10:56
  • If you want to know which of the three is called first? They're not all called IIRC at same time in one call, they're called under different circomstances.. see more at http://docs.python.org/2/reference/datamodel.html – Torxed Feb 09 '13 at 11:11

4 Answers4

47

It's a bit complicated. Here's the sequence of checks Python does if you request an attribute of an object.

First, Python will check if the object's class has a __getattribute__ method. If it doesn't have one defined, it will inherit object.__getattribute__ which implements the other ways of finding the attribute's values.

The next check is in the object's class's __dict__. However, even if a value is found there, it may not be the result of the attribute lookup! Only "data descriptors" will take precedence if found here. The most common data descriptor is a property object, which is a wrapper around a function that will be called each time an attribute is accessed. You can create a property with a decorator:

class foo(object):
    @property
    def myAttr(self):
        return 2

In this class, myAttr is a data descriptor. That simply means that it implements the descriptor protocol by having both __get__ and __set__ methods. A property is a data descriptor.

If the class doesn't have anything in its __dict__ with the requested name, object.__getattribute__ searches through its base classes (following the MRO) to see if one is inherited. An inherited data descriptor works just like one in the object's class.

If a data descriptor was found, its __get__ method is called and the return value becomes the value of the attribute lookup. If an object that is not a data descriptor was found, it is held on to for a moment, but not returned just yet.

Next, the object's own __dict__ is checked for the attribute. This is where most normal member variables are found.

If the object's __dict__ didn't have anything, but the earlier search through the class (or base classes) found something other than a data descriptor, it takes next precedence. An ordinary class variable will be simply returned, but "non-data descriptors" will get a little more processing.

A non-data descriptor is an object with a __get__ method, but no __set__ method. The most common kinds of non-data descriptors are functions, which become bound methods when accessed as a non-data descriptor from an object (this is how Python can pass the object as the first argument automatically). The descriptor's __get__ method will be called and it's return value will be the result of the attribute lookup.

Finally, if none of the previous checks succeeded, __getattr__ will be called, if it exists.

Here are some classes that use steadily increasing priority attribute access mechanisms to override the behavior of their parent class:

class O1(object):
    def __getattr__(self, name):
        return "__getattr__ has the lowest priority to find {}".format(name)

class O2(O1):
    var = "Class variables and non-data descriptors are low priority"
    def method(self): # functions are non-data descriptors
        return self.var

class O3(O2):
    def __init__(self):
        self.var = "instance variables have medium priority"
        self.method = lambda: self.var # doesn't recieve self as arg

class O4(O3):
    @property # this decorator makes this instancevar into a data descriptor
    def var(self):
        return "Data descriptors (such as properties) are high priority"

    @var.setter # I'll let O3's constructor set a value in __dict__
    def var(self, value):
        self.__dict__["var"]  = value # but I know it will be ignored

class O5(O4):
    def __getattribute__(self, name):
        if name in ("magic", "method", "__dict__"): # for a few names
            return super(O5, self).__getattribute__(name) # use normal access
        
        return "__getattribute__ has the highest priority for {}".format(name)

And, a demonstration of the classes in action:

O1 (__getattr__):

>>> o1 = O1()
>>> o1.var
'__getattr__ has the lowest priority to find var'

O2 (class variables and non-data descriptors):

>>> o2 = O2()
>>> o2.var
'Class variables and non-data descriptors are low priority'
>>> o2.method
<bound method O2.method of <__main__.O2 object at 0x000000000338CD30>>
>>> o2.method()
'Class variables and non-data descriptors are low priority'

O3 (instance variables, including a locally overridden method):

>>> o3 = O3()
>>> o3.method
<function O3.__init__.<locals>.<lambda> at 0x00000000034AAEA0>
>>> o3.method()
'instance variables have medium priority'
>>> o3.var
'instance variables have medium priority'

O4 (data descriptors, using the property decorator):

>>> o4 = O4()
>>> o4.method()
'Data descriptors (such as properties) are high priority'
>>> o4.var
'Data descriptors (such as properties) are high priority'
>>> o4.__dict__["var"]
'instance variables have medium priority'

O5 (__getattribute__):

>>> o5 = O5()
>>> o5.method
<function O3.__init__.<locals>.<lambda> at 0x0000000003428EA0>
>>> o5.method()
'__getattribute__ has the highest priority for var'
>>> o5.__dict__["var"]
'instance variables have medium priority'
>>> o5.magic
'__getattr__ has the lowest priority to find magic'
urig
  • 16,016
  • 26
  • 115
  • 184
Blckknght
  • 100,903
  • 11
  • 120
  • 169
  • Can you give me the example code like Torx has given and then using `Print "function name" rasie AttributeError` with each function so that i can see the chain of events – user196264097 Feb 09 '13 at 12:03
  • That's a bit tricky. Only one function will be called by any give attribute lookup. Which one depends on the precedence I've detailed above. I think I may have been editing while you posted your request, so look the answer over again, and it may be clearer. – Blckknght Feb 09 '13 at 12:10
  • I am trying the modified version of Totx code here http://pastebin.com/rtrpCGQy. i can get into `Getattribute` first then it enters `getattr` and then it raise error. why i didn't went tnto `__dict` – user196264097 Feb 09 '13 at 12:27
  • @user9: I've updated with a bunch of example code and demonstrations of the different kinds of member access. Each class overrides the behavior of the previous one. – Blckknght Feb 09 '13 at 13:44
  • How about setting attributes? How do I make it so that python looks if there is an attribute with that name and if not calls `__setattr__`? – Nearoo Jun 16 '17 at 14:39
  • The logic for assigning to an attribute is a bit simpler than it is for lookups. There are three steps. The first thing checked is whether a `__setattr__` method exists in the class. If it does, it's always called. If not, the second step is to check if the attribute describes a data descriptor in the class (or an inherited base class). If a data descriptor is found, its `__set__` method is called. Non-data descriptors and other class variables are ignored (they will be shadowed). If neither of the previous cases were hit, the value is simply assigned into the instance's `__dict__`. – Blckknght Jun 16 '17 at 21:26
1
class test():
    def __init__(self):
        self.a = 1

    def __getattribute__(self, attr):
        print 'Getattribute:',attr

    def __getattr__(self, attr):
        print 'GetAttr:',attr

    def __dict__(self, attr):
        print 'Dict:',attr

    def __call__(self, args=None):
        print 'Called:',args

    def __getitem__(self, attr):
        print 'GetItem:',attr

    def __get__(self, instance, owner):
        print 'Get:',instance,owner

    def __int__(self):
        print 'Int'



x = test()
print x.a

None of the above will be called..

[root@faparch doxid]# python -m trace --trace test_dict.py
 --- modulename: test_dict, funcname: <module>
test_dict.py(1): class test():
 --- modulename: test_dict, funcname: test
test_dict.py(1): class test():
test_dict.py(2):    def __init__(self):
test_dict.py(5):    def __getattribute__(self, attr):
test_dict.py(8):    def __getattr__(self, attr):
test_dict.py(11):   def __dict__(self, attr):
test_dict.py(14):   def __call__(self, args=None):
test_dict.py(17):   def __getitem__(self, attr):
test_dict.py(20):   def __get__(self, instance, owner):
test_dict.py(23):   def __int__(self):
test_dict.py(28): x = test()
 --- modulename: test_dict, funcname: __init__
test_dict.py(3):        self.a = 1
test_dict.py(29): print x.a
1
 --- modulename: trace, funcname: _unsettrace
trace.py(80):         sys.settrace(None)

You'd might want to look into: http://docs.python.org/2/library/numbers.html#numbers.Number Most likely you'll need to implement a nested class that handles the number-class functions in order to snatch up calls such as in the example. Or, at least that's one way of doing it..

The integer value contains the following functions which you have to intercept

['__abs__', '__add__', '__and__', '__class__', '__cmp__', '__coerce__', '__delattr__', '__div__', '__divmod__', '__doc__', '__float__', '__floordiv__', '__format__', '__getattribute__', '__getnewargs__', '__hash__', '__hex__', '__index__', '__init__', '__int__', '__invert__', '__long__', '__lshift__', '__mod__', '__mul__', '__neg__', '__new__', '__nonzero__', '__oct__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdiv__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'bit_length', 'conjugate', 'denominator', 'imag', 'numerator', 'real']
Torxed
  • 22,866
  • 14
  • 82
  • 131
  • 1
    Was your test done in Python 2? If so, you should make your class inherit from `object`, or you'll get an old-style class that doesn't use the `__getattribute__` method (or descriptors, or many of the other complicated bits of Python's data model). – Blckknght Feb 09 '13 at 11:42
  • I tried inheriting from `Object` and then `raise AttributeError` then i get this `Getattribute: a GetAttr: a None` . how can i go through other chain methods. I didn't went past __getattr__. – user196264097 Feb 09 '13 at 12:01
  • It's because in my example, i never return any values, hence the `None` output. You'll need to intercept `__setitem__` etc as well and keep track somehow what values are set and return them accordingly :) – Torxed Feb 09 '13 at 12:12
0

The namespace dictionary(__dict__) gets called first.

From the docs:

A class instance has a namespace implemented as a dictionary which is the first place in which attribute references are searched. When an attribute is not found there, and the instance’s class has an attribute by that name, the search continues with the class attributes If no class attribute is found, and the object’s class has a __getattr__() method, that is called to satisfy the lookup.

Docs on object.__getattribute__:

Called unconditionally to implement attribute accesses for instances of the class. If the class also defines __getattr__(), the latter will not be called unless __getattribute__() either calls it explicitly or raises an AttributeError. This method should return the (computed) attribute value or raise an AttributeError exception.

Docs on object.__getattr__(self, name):

If the attribute is found through the normal mechanism, __getattr__() is not called.

Ashwini Chaudhary
  • 244,495
  • 58
  • 464
  • 504
0

From the docs:

... For example, obj.d looks up d in the dictionary of obj. If d defines the method __get__(), then d.__get__(obj) is invoked according to the precedence rules listed below.

... For objects, the machinery is in object.__getattribute__() which transforms b.x into type(b).__dict__['x'].__get__(b, type(b)). The implementation works through a precedence chain that gives data descriptors priority over instance variables, instance variables priority over non-data descriptors, and assigns lowest priority to __getattr__() if provided.

That is, obj.a calls __getattribute__() that by default uses __dict__. __getattr__() is called as the last resort. The rest describes descriptors e.g., property or ordinary methods behavior.

how can d define the method __get__()

import random 

class C(object):
    @property
    def d(self):
        return random.random()

c = C()
print(c.d)
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • how can `d` define the method `__get__()`. can you give example – user196264097 Feb 09 '13 at 11:20
  • @user9: I've added an example – jfs Feb 09 '13 at 11:27
  • but u didn't define `__get__` method. Also Torex is saying that nothing is called. Is he right. Can you give me examlike like torx where i can see all the print statement so that i can see the chain of calling methods – user196264097 Feb 09 '13 at 11:42
  • @user9: Torex's answer uses old-style class (it is not an `object`'s subclass). The docs shows [pure Python `property` implementation](http://docs.python.org/2/howto/descriptor.html#properties) where you can see how `__get__()` could be defined. You don't need to know about descriptors to understand a simple attribute access as in your question. – jfs Feb 09 '13 at 12:09
  • I am trying the modified version of Torx code here http://pastebin.com/rtrpCGQy. i can get into `Getattribute` first then it enters `getattr` and then it raise error. why i didn't went tnto `__dict`a – user196264097 Feb 09 '13 at 12:28
  • @user9: `__dict__` is a dictionary (it is defined implicitly), not a method. – jfs Feb 09 '13 at 13:06