AttributeErrors: undesired interaction between @property and getattr

Question

I have a problem with AttributeErrors raised in a @property in combination with __getattr__() in python:

Example code:

>>> def deeply_nested_factory_fn():
...     a = 2
...     return a.invalid_attr
...
>>> class Test(object):
...     def __getattr__(self, name):
...         if name == 'abc':
...             return 'abc'
...         raise AttributeError("'Test' object has no attribute '%s'" % name)
...     @property
...     def my_prop(self):
...         return deeply_nested_factory_fn()
...
>>> test = Test()
>>> test.my_prop
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in __getattr__
AttributeError: 'Test' object has no attribute 'my_prop'

In my case, this is a highly misleading error message, because it hides the fact that deeply_nested_factory_fn() has a mistake.

Based on the idea in Tadhg McDonald-Jensen's answer, my currently best solution is the following. Any hints on how to get rid of the __main__. prefix to AttributeError and the reference to attributeErrorCatcher in the traceback would be much appreciated.

>>> def catchAttributeErrors(func):
...     AttributeError_org = AttributeError
...     def attributeErrorCatcher(*args, **kwargs):
...         try:
...             return func(*args, **kwargs)
...         except AttributeError_org as e:
...             import sys
...             class AttributeError(Exception):
...                 pass
...             etype, value, tb = sys.exc_info()
...             raise AttributeError(e).with_traceback(tb.tb_next) from None
...     return attributeErrorCatcher
...
>>> def deeply_nested_factory_fn():
...     a = 2
...     return a.invalid_attr
...
>>> class Test(object):
...     def __getattr__(self, name):
...         if name == 'abc':
...             # computing come other attributes
...             return 'abc'
...         raise AttributeError("'Test' object has no attribute '%s'" % name)
...     @property
...     @catchAttributeErrors
...     def my_prop(self):
...         return deeply_nested_factory_fn()
...
>>> class Test1(object):
...     def __init__(self):
...         test = Test()
...         test.my_prop
...
>>> test1 = Test1()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in __init__
  File "<stdin>", line 11, in attributeErrorCatcher
  File "<stdin>", line 10, in my_prop
  File "<stdin>", line 3, in deeply_nested_factory_fn
__main__.AttributeError: 'int' object has no attribute 'invalid_attr'

you would have to do `__qualname__ = "AttributeError"` in the class definition to remove the `__main__` part but trust me, you do **not** want the error to just say AttributeError because then you risk being baffled why the heck `except AttributeError` didn't catch an AttributeError. — Tadhg McDonald-Jensen, Apr 12 '16 at 21:42

score 3 · Answer 1 · answered Apr 12 '16 at 13:49

3

If you're willing to exclusively use new-style classes, you could overload __getattribute__ instead of __getattr__:

class Test(object):
    def __getattribute__(self, name):
        if name == 'abc':
            return 'abc'
        else:
            return object.__getattribute__(self, name)
    @property
    def my_prop(self):
        return deeply_nested_factory_fn()

Now your stack trace will properly mention deeply_nested_factory_fn.

Traceback (most recent call last):
  File "C:\python\myprogram.py", line 16, in <module>
    test.my_prop
  File "C:\python\myprogram.py", line 10, in __getattribute__
    return object.__getattribute__(self, name)
  File "C:\python\myprogram.py", line 13, in my_prop
    return deeply_nested_factory_fn()
  File "C:\python\myprogram.py", line 3, in deeply_nested_factory_fn
    return a.invalid_attr
AttributeError: 'int' object has no attribute 'invalid_attr'

answered Apr 12 '16 at 13:49

Kevin

74,910
12
133
166

Thanks, my issue with this solution is the performance hit for other attributes. (Some of which may be accessed in loops.) – ARF Apr 12 '16 at 13:51
@ARF, if performance is a concern then get local references to the frequently accessed attributes. I'd recommend doing that even _without_ the slight slowing down by overloading `__getattribute__`. – Tadhg McDonald-Jensen Apr 12 '16 at 15:34
You mean by `a = test.a` and then using `a` instead of `test.a`? – ARF Apr 12 '16 at 20:03
@ARF yes, I actually advertised Kevin's answer in mine because it is by far superior to mine. – Tadhg McDonald-Jensen Apr 12 '16 at 21:45

score 1 · Answer 2 · edited May 23 '17 at 10:28

You can create a custom Exception that appears to be an AttributeError but will not trigger __getattr__ since it is not actually an AttributeError.

UPDATED: the traceback message is greatly improved by reassigning the .__traceback__ attribute before re-raising the error:

class AttributeError_alt(Exception):
    @classmethod
    def wrapper(err_type, f):
        """wraps a function to reraise an AttributeError as the alternate type"""
        @functools.wraps(f)
        def alt_AttrError_wrapper(*args,**kw):
            try:
                return f(*args,**kw)
            except AttributeError as e:
                new_err = err_type(e)
                new_err.__traceback__ = e.__traceback__.tb_next
                raise new_err from None
        return alt_AttrError_wrapper

Then when you define your property as:

@property
@AttributeError_alt.wrapper
def my_prop(self):
    return deeply_nested_factory_fn()

and the error message you will get will look like this:

Traceback (most recent call last):
  File ".../test.py", line 34, in <module>
    test.my_prop
  File ".../test.py", line 14, in alt_AttrError_wrapper
    raise new_err from None
  File ".../test.py", line 30, in my_prop
    return deeply_nested_factory_fn()
  File ".../test.py", line 20, in deeply_nested_factory_fn
    return a.invalid_attr
AttributeError_alt: 'int' object has no attribute 'invalid_attr'

notice there is a line for raise new_err from None but it is above the lines from within the property call. There would also be a line for return f(*args,**kw) but that is omitted with .tb_next.

I am fairly sure the best solution to your problem has already been suggested and you can see the previous revision of my answer for why I think it is the best option. Although honestly if there is an error that is incorrectly being suppressed then raise a bloody RuntimeError chained to the one that would be hidden otherwise:

def assert_no_AttributeError(f):
    @functools.wraps(f)
    def assert_no_AttrError_wrapper(*args,**kw):
        try:
            return f(*args,**kw)
        except AttributeError as e:
            e.__traceback__ = e.__traceback__.tb_next
            raise RuntimeError("AttributeError was incorrectly raised") from e
    return assert_no_AttrError_wrapper

then if you decorate your property with this you will get an error like this:

Traceback (most recent call last):
  File ".../test.py", line 27, in my_prop
    return deeply_nested_factory_fn()
  File ".../test.py", line 17, in deeply_nested_factory_fn
    return a.invalid_attr
AttributeError: 'int' object has no attribute 'invalid_attr'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".../test.py", line 32, in <module>
    x.my_prop
  File ".../test.py", line 11, in assert_no_AttrError_wrapper
    raise RuntimeError("AttributeError was incorrectly raised") from e
RuntimeError: AttributeError was incorrectly raised

Although if you expect more then just one thing to raise an AttributeError then you might want to just overload __getattribute__ to check for any peculiar error for all lookups:

def __getattribute__(self,attr):
    try:
        return object.__getattribute__(self,attr)
    except AttributeError as e:
        if str(e) == "{0.__class__.__name__!r} object has no attribute {1!r}".format(self,attr):
            raise #normal case of "attribute not found"
        else: #if the error message was anything else then it *causes* a RuntimeError
            raise RuntimeError("Unexpected AttributeError") from e

This way when something goes wrong that you are not expecting you will know it right away!

This does not really make things much better: The stack trace is still way off! Note that the error is is in function `deeply_nested_factory_fn` not in function `inner` as suggested by the stack trace produced by your solution. - That said, I would really like your idea which avoids `sys.exit(1)`... if the stacktrace issue could be fixed. — ARF, Apr 12 '16 at 21:00
After some head-scratching, I came up with a solution I am not too unhappy about. If you are still interested, have a look as the modified question. — ARF, Apr 12 '16 at 21:25
This is going to stop people from doing `except AttributeError` to catch this exception. — user2357112, Apr 12 '16 at 21:29
@user2357112 That is an excellent point. I will have to think about things a bit more. — ARF, Apr 12 '16 at 21:45
@TadhgMcDonald-Jensen Actually, for my application the `hasattr` behaviour you illustrate is desirable. I am having the issue that `hasattr` fails "silently" in a `if hasattr(...): .... else: .... ` block - not because the property is undefined but because the implementation has a bug. Of course that other piece of code should really avoid `hasattr` altogether and just use a `try: some_obj.attr except AttributeError: ....` block instead... — ARF, Apr 12 '16 at 21:47
@TadhgMcDonald-Jensen To avoid the issue alltogether it would be nice if one could "re-raise" the attribute error in `__getattr__` depending on whether the `AttributeError` originated in the property on not. But I think, once we get to `__getattr__` all traces of the `AttributeError` from the property are gone. Or is there a way to recover this error and its stack trace somehow? — ARF, Apr 12 '16 at 21:56
I edited my answer to suggest exception chaining just like I did with [this question](http://stackoverflow.com/questions/34175111/raise-an-exception-from-a-higher-level-a-la-warnings). And I'll tell you what I told him: Faking a seamless error message is **not** something you want to do. You want an error that tells you **what actually happened** in execution and be happy that python gives such informative traceback messages. — Tadhg McDonald-Jensen, Apr 13 '16 at 02:59
Thanks for the RuntimeError chaining suggestion. It looks like a simple, clean solution for my particular situation. — ARF, Apr 13 '16 at 20:20

score 1 · Accepted Answer · answered Dec 04 '22 at 17:14

Just in case others find this: the problem with the example on top is that an AttributeError is raised inside __getattr__. Instead, one should call self.__getattribute__(attr) to let that raise.

Example

def deeply_nested_factory_fn():
    a = 2
    return a.invalid_attr

class Test(object):
    def __getattr__(self, name):
        if name == 'abc':
            return 'abc'
        return self.__getattribute__(name)
    @property
    def my_prop(self):
        return deeply_nested_factory_fn()

test = Test()
test.my_prop

This yields

AttributeError                            Traceback (most recent call last)
Cell In [1], line 15
     12         return deeply_nested_factory_fn()
     14 test = Test()
---> 15 test.my_prop

Cell In [1], line 9, in Test.__getattr__(self, name)
      7 if name == 'abc':
      8     return 'abc'
----> 9 return self.__getattribute__(name)

Cell In [1], line 12, in Test.my_prop(self)
     10 @property
     11 def my_prop(self):
---> 12     return deeply_nested_factory_fn()

Cell In [1], line 3, in deeply_nested_factory_fn()
      1 def deeply_nested_factory_fn():
      2     a = 2
----> 3     return a.invalid_attr

AttributeError: 'int' object has no attribute 'invalid_attr'

AttributeErrors: undesired interaction between @property and __getattr__

3 Answers3

AttributeErrors: undesired interaction between @property and getattr