Adding custom objects ignores __getattr__ with operators

Question

I am trying to create a custom object that passes all non-existent method calls down to a member attribute. This works under normal custom method invocations, but fails when attempting to call arithmetic operators.

Below is a console snippet of an example class, a test function, and a cleaned up disassembly of the test function.

>>> class NoAdd(object):
...    member = 0
...    def __getattr__(self, item):
...        print('NoAdd __getattr__')
...        # return getattr(self.member, item)
...        if item == '__add__':
...            return self.member.__add__
>>> def test():
...    print('Call __add__ directly.')
...    NoAdd().__add__(5)  # 5
...    print('Implicit __add__.')
...    NoAdd() + 5  # TypeError
>>> dis(test)
  3           8 LOAD_GLOBAL              1 (NoAdd)
             10 CALL_FUNCTION            0
             12 LOAD_ATTR                2 (__add__)
             14 LOAD_CONST               2 (5)
             16 CALL_FUNCTION            1
             18 POP_TOP
  5          28 LOAD_GLOBAL              1 (NoAdd)
             30 CALL_FUNCTION            0
             32 LOAD_CONST               2 (5)
             34 BINARY_ADD
             36 POP_TOP
             38 LOAD_CONST               0 (None)
             40 RETURN_VALUE
>>> test()
Call __add__ directly.
NoAdd __getattr__
Implicit __add__.
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "<input>", line 5, in test
TypeError: unsupported operand type(s) for +: 'NoAdd' and 'int'

I thought that the Python interpreter would look for the __add__ method using the standard procedure of invoking __getattr__ when the method was not found in the object's method list, before looking for __radd__ in the int. This is apparently not what is happening.

Can someone help me out by explaining what is going on or helping me find out where in the Python source code I can find what BINARY_ADD is doing? I'm not sure if I can fix this, but a workaround would be appreciated.

Answer 1

The methods __getattr__ and __getattribute__ are only used for recovering attributes when you call then explicitly, by example when you do foo.bar. They are not used for implicit invocation.

This behaviour is specified in the documentation

Note: This method may still be bypassed when looking up special methods as the result of implicit invocation via language syntax or built-in functions.

The reason for such an implementation is explained here.

Bypassing the __getattribute__() machinery in this fashion provides significant scope for speed optimisations within the interpreter, at the cost of some flexibility in the handling of special methods

In conclusion what you are trying to do, i.e. using __getattr__ to redirect implicit special method calls, has been voluntarily sacrificed in favor of speed.

Inheritance

The behaviour you are describing can be achieved by class inheritance. Although, passing in arguments to your class constructor will require the following fidling with the __new__ method.

class NoAdd(int):

    def __new__(cls, x, *args, **kwargs):
        return super().__new__(cls, x)

    def __init__(self, x, *args, **kwargs):
        ...

x = NoAdd(0)

x + 5 # 5
x * 2 # 0

Metaclass

Now, suppose you really need to catch implicit call to special methods. I see very little case where this could be useful, but it is a fun exercise. In this case we can rely on metaclass to fill in missing methods with the ones from member.

class ProxyToMember(type):

    def __init__(cls, name, bases, name_space):
        super().__init__(name, bases, name_space)

        if hasattr(cls, 'member'):
            proxy_attrs = (attr for attr in dir(cls.member) if not hasattr(cls, attr))

            def make_proxy(attr):

                attr_value = getattr(cls.member, attr)

                def proxy(_, *args, **kwargs):
                    return attr_value(*args, **kwargs)

                if callable(attr_value):
                    return proxy
                else:
                    return property(lambda _: getattr(cls.member, attr))

            for attr in proxy_attrs:
                setattr(cls, attr, make_proxy(attr))

class A(metaclass=ProxyToMember):
    member = 0

class B(metaclass=ProxyToMember):
    member = 'foo'

A() + 1 # 1
B().startswith('f') # True

Answer 2

I don't understand you.

Why you can't do something like:

class MyInt(int):
    pass

def test():
    print(MyInt(5) + 5)

test()

Answer 3

After some more intense research (and following some initially unlikely trails) I found my answer. My thought process didn't come up with the same search keywords, which is why I didn't find these before. My question could probably be considered a duplicate of one of the ones linked below.

I found the simplest/best explanation for why this happens here. I found some good references for resolving this here and here. The links in Oliver's answer are also helpful.

In summary, Python does not use the standard method lookup process for the magic methods such as __add__ and __str__. It looks like the workaround is to make a Wrapper class from here.