understanding getattribute and pdb.set_trace() with a Python code snippet as an example

Question

I have read questions like this on SO and I believe I understand the danger of using __getattribute__. However, recently I took over a project from others and I need to make some modifications. I think the best way to understand a project is by tracing it--so I inserted pdb.set_trace() and then I pressed "n\r\n"--but the program did not get executed to next line and wait for new input but instead continue execution all the way to its end. After searching, I think it is the misuse of __getattribute__ causing the problem but I do not know why. I simplify the code to the following:

class TestAttribute(object):
    """docstring for TestAttribute"""
    def __init__(self, is_testing=False):
        super(TestAttribute, self).__init__()
        self.is_testing = is_testing

    def __getattribute__(self, name):
        # print(name)
        try:
            # the line below will trigger the recursion error
            if self.is_testing:
                name = name.upper()
            return super(TestAttribute, self).__getattribute__(name)
        except AttributeError:
            return None
        except Exception:
            # this line is added by me to see the output
            import traceback; traceback.print_exc();
            return None

    def __getitem__(self, name):
        return self.__getattribute__(name)

    def __setitem__(self, name, val):
        return self.__setattr__(name, val)

    def __setattr__(self, name, val):
        # so this func will be called in __init__ and will
        # enter __getattribute__
        if self.is_testing:
            name = name.lower()
        super(TestAttribute, self).__setattr__(name, val)


if __name__ == '__main__':
    ttt = TestAttribute()
    import pdb; pdb.set_trace()
    ttt.k = 1
    print('test done')
    print('test done again')
    print('test done again')
    print('test done again')

Output as below:

Traceback (most recent call last):
  File "test_getattribute.py", line 10, in __getattribute__
Traceback (most recent call last):
  File "test_getattribute.py", line 10, in __getattribute__
    if self.is_testing:
  File "test_getattribute.py", line 16, in __getattribute__
    import traceback; traceback.print_exc();
  File "/usr/lib/python2.7/traceback.py", line 232, in print_exc
    print_exception(etype, value, tb, limit, file)
  File "/usr/lib/python2.7/traceback.py", line 125, in print_exception
    print_tb(tb, limit, file)
  File "/usr/lib/python2.7/traceback.py", line 69, in print_tb
    line = linecache.getline(filename, lineno, f.f_globals)
  File "/home/jgu/repos/.venv/lib/python2.7/linecache.py", line 14, in getline
    lines = getlines(filename, module_globals)
  File "/home/jgu/repos/.venv/lib/python2.7/linecache.py", line 40, in getlines
    return updatecache(filename, module_globals)
RuntimeError: maximum recursion depth exceeded
> /home/jgu/repos/dat_cs/test_getattribute.py(34)<module>()
-> ttt.k = 1
(Pdb) n
Traceback (most recent call last):
  File "test_getattribute.py", line 10, in __getattribute__
    if self.is_testing:
  File "test_getattribute.py", line 7, in __getattribute__
    def __getattribute__(self, name):
  File "/usr/lib/python2.7/bdb.py", line 50, in trace_dispatch
    return self.dispatch_call(frame, arg)
  File "/usr/lib/python2.7/bdb.py", line 76, in dispatch_call
    if not (self.stop_here(frame) or self.break_anywhere(frame)):
  File "/usr/lib/python2.7/bdb.py", line 147, in break_anywhere
    return self.canonic(frame.f_code.co_filename) in self.breaks
  File "/usr/lib/python2.7/bdb.py", line 29, in canonic
    if filename == "<" + filename[1:-1] + ">":
RuntimeError: maximum recursion depth exceeded in cmp
test done
test done again
test done again
test done again

As you can see, I pressed "n\r\n" only and the execution continues all the way to program finishes.

And there is also another small question, if I run without pdb, I see this output:

Traceback (most recent call last):
  File "test_getattribute.py", line 10, in __getattribute__
Traceback (most recent call last):
  File "test_getattribute.py", line 10, in __getattribute__
    if self.is_testing:
  File "test_getattribute.py", line 16, in __getattribute__
    import traceback; traceback.print_exc();
  File "/usr/lib/python2.7/traceback.py", line 232, in print_exc
    print_exception(etype, value, tb, limit, file)
  File "/usr/lib/python2.7/traceback.py", line 125, in print_exception
    print_tb(tb, limit, file)
  File "/usr/lib/python2.7/traceback.py", line 69, in print_tb
    line = linecache.getline(filename, lineno, f.f_globals)
  File "/home/jgu/repos/.venv/lib/python2.7/linecache.py", line 14, in getline
    lines = getlines(filename, module_globals)
  File "/home/jgu/repos/.venv/lib/python2.7/linecache.py", line 40, in getlines
    return updatecache(filename, module_globals)
RuntimeError: maximum recursion depth exceeded
Traceback (most recent call last):
Traceback (most recent call last):
  File "test_getattribute.py", line 10, in __getattribute__
    if self.is_testing:
  File "test_getattribute.py", line 16, in __getattribute__
    import traceback; traceback.print_exc();
  File "/usr/lib/python2.7/traceback.py", line 232, in print_exc
    print_exception(etype, value, tb, limit, file)
  File "/usr/lib/python2.7/traceback.py", line 125, in print_exception
    print_tb(tb, limit, file)
  File "/usr/lib/python2.7/traceback.py", line 67, in print_tb
    '  File "%s", line %d, in %s' % (filename, lineno, name))
RuntimeError: <unprintable RuntimeError object>
test done
test done again
test done again
test done again

So the second error is not printed correctly, why is this?

Edit: I am not asking why I get recursion error. I believe I am clear on that part--so please understand my question first. Thanks

`if self.is_testing` - well, how do you find the value of `self.is_testing`? You call `__getattribute__`. To execute `__getattribute__`, the first thing you do is check `self.is_testing`. How do you find the value of `self.is_testing`? You call `__getattribute__` ... — user2357112, Dec 15 '16 at 07:28
I am not asking why I get recursion error. I think I am clear on that part. — Junchao Gu, Dec 15 '16 at 07:33

user2357112 · Accepted Answer · 2016-12-15T08:25:17.127

2

pdb uses sys.settrace to set a trace function to do its job. Any exception that propagates out of the trace function disables it. The RuntimeError happens inside the trace function, essentially turning off pdb once it happens.

edited Dec 15 '16 at 08:25

answered Dec 15 '16 at 07:49

user2357112

260,549
28
431
505

got it. thanks for the explanation. what about the second question? just wonder if you can help regarding that as well – Junchao Gu Dec 15 '16 at 08:11
@JunchaoGu: The stack is still crazy deep when you catch the RuntimeError, and you get *another* stack overflow while trying to display the stack trace, in a location that happens to trigger a [certain fallback](https://hg.python.org/cpython/file/2.7/Lib/traceback.py#l212). – user2357112 Dec 15 '16 at 08:29
By the way, you really shouldn't have caught those exceptions in `__getattribute__`. – user2357112 Dec 15 '16 at 08:30

understanding __getattribute__ and pdb.set_trace() with a Python code snippet as an example

1 Answers1

understanding getattribute and pdb.set_trace() with a Python code snippet as an example