Method overriding by monkey patching

Question

I'm trying to figure out why the following example won't work.

class BaseClass(object):
    def __init__(self):
        self.count = 1

    def __iter__(self):
        return self

    def next(self):
        if self.count:
            self.count -= 1
            return self
        else:
            raise StopIteration


class DerivedNO(BaseClass):
    pass


class DerivedO(BaseClass):
    def __init__(self):
        self.new_count = 2
        self.next = self.new_next

    def new_next(self):
        if self.new_count:
            self.new_count -= 1
            return None
        else:
            raise StopIteration


x = DerivedNO()
y = DerivedO()

print x
print list(x)
print y
print list(y)

And here is the output:

<__main__.DerivedNO object at 0x7fb2af7d1c90>
[<__main__.DerivedNO object at 0x7fb2af7d1c90>]
<__main__.DerivedO object at 0x7fb2af7d1d10>
Traceback (most recent call last):
  File "playground.py", line 41, in <module>
    print list(y)
  File "playground.py", line 11, in next
    if self.count:
AttributeError: 'DerivedO' object has no attribute 'count'

As you can see the new method will not be overridden in DerivedO when I try to assign the next() method in __init__. Why is that? A simple call to next will work fine, but not at all when using iterating techniques.

Edit: I realize my question wasn't completely clear. The AttributeError isn't the problem I'm looking to solve. But it does show that next() is called on BaseClass instead of on DerivedO as I thought it would.

I think you just need to call the base class `__init__` in your derived class constructor. — sje397, Jul 27 '12 at 12:23
@sje397: that's the point; the `new_next` method does not refer to `count`. — Martijn Pieters, Jul 27 '12 at 12:29
Interesting, when returning `self` from `__iter__` the class-defined `next` is used, whatever `next` may have been set to on the instance. The same applies to the `__iter__` method, patching it with a method using `yield` is also ignored. — Martijn Pieters, Jul 27 '12 at 12:34
possible duplicate of [Python `__iter__` and for loops](http://stackoverflow.com/questions/5300988/python-iter-and-for-loops) — Martijn Pieters, Jul 27 '12 at 12:39

Martijn Pieters · Accepted Answer · 2012-07-27T12:52:35.683

8

You cannot monkeypatch either __iter__(self) or, by extension, next(self) on instances because these methods are treated as class methods instead as a CPython internal optimization (see Special method lookup for new-style classes for an in-depth rationale as to why this is).

If you need to monkeypatch these methods, you'll need to set them directly on the class instead:

class DerivedO(BaseClass):
    def __init__(self):
        self.new_count = 2
        self.__class__.next = self.__class__.new_next

    def new_next(self):
        if self.new_count:
            self.new_count -= 1
            return None
        else:
            raise StopIteration

The above will work; note that I set __class__.next to the unbound function new_next, not to the bound method.

edited Jul 27 '12 at 12:52

answered Jul 27 '12 at 12:44

Martijn Pieters

1,048,767
296
4,058
3,343

Ah I see, thank you. Everytime I think I know Python it always turns out there's a lot more to it! – Fulkerson Jul 27 '12 at 12:52
This will work, but it's worth mentioning that at this point, you might as well just override the method using standard syntax (i.e. by putting `next = new_next` after the definition of `new_next`). The approach you use here is rather obfuscated! – senderle Jul 27 '12 at 12:55
@senderle: I was assuming the OP had constructed an example to illustrate his problem. The monkeypatching could also happen on classes outside of his or her control, at which point the theory behind my answer still applies. – Martijn Pieters Jul 27 '12 at 12:57
@MartijnPieters, OK, I see what you mean. I just thought it was worth mentioning as a caveat. – senderle Jul 27 '12 at 12:59
Yeah, the idea was to select a lean next function during init depending on certain runtime information instead of having to go through a lot of ifs and elifs each iteration. But it's getting overly complex so I think I'll go for another approach instead. Mainly I was annoyed it didn't work the way I thought it should. – Fulkerson Jul 27 '12 at 13:15

unutbu · Answer 2 · 2012-07-27T12:29:03.517

-1

Since DerivedO never initializes the count attribute, an AttributeError occurs when the next method is executed.

You could avoid this error by arranging for BaseClass.__init__ to be called (either explicitly, or by using super):

class DerivedO(BaseClass):
    def __init__(self):
        super(DerivedO, self).__init__()
        self.new_count = 2

    def next(self):
        if self.new_count:
            self.new_count -= 1
            return None
        else:
            raise StopIteration

Also, instead of defining new_next, you could simply override (redefine) next.

edited Jul 27 '12 at 12:29

answered Jul 27 '12 at 12:24

unutbu

842,883
184
1,785
1,677

2

That's it though; the OP has an interesting question. Why can't you *monkeypatch* next? The iterator uses the class-defined `next` method, not the instance override. You are not addressing the real issue here. – Martijn Pieters Jul 27 '12 at 12:32

Method overriding by monkey patching

2 Answers2