19

Is there a way to make len() work with instance methods without modifying the class?

Example of my problem:

>>> class A(object):
...     pass
...
>>> a = A()
>>> a.__len__ = lambda: 2
>>> a.__len__()
2
>>> len(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object of type 'A' has no len()

Note:

  • different instances of A will have different __len__ methods attached
  • I cannot change the class A
smci
  • 32,567
  • 20
  • 113
  • 146
ARF
  • 7,420
  • 8
  • 45
  • 72
  • 1
    Your problem is **when** you're defining `__len__`, try defining it as an actual method on the class instead of an attribute of an instance. – jesseops Apr 11 '16 at 19:07
  • 1
    @PadraicCunningham See my modified question. This does not allow different `__len__` methods for different instances of `A`. – ARF Apr 11 '16 at 19:07
  • 1
    @ARF, I presume you cannot modify the class? – Padraic Cunningham Apr 11 '16 at 19:09
  • 1
    @PadraicCunningham You are right, I cannot modify the class. Based on the answers I am getting, I now see that I really should have mentioned this. – ARF Apr 11 '16 at 19:22
  • 1
    @ARF: That sounds like you should either have your own wrapper type that contains an `A` object, or you should inherit from `A`. I recommend the wrapper; inheritance is deceptively tricky, especially when the superclass isn't designed for it. – user2357112 Apr 11 '16 at 19:26
  • 1
    Can you shed some light on what kind of trick is used for ? – Fruch Apr 17 '16 at 06:34
  • Why would you ever wan to do this? Is it for testing, or some other reason? Given you can't modify the class, why don't you use a wrapper type that modifies `inst.__len__`? – smci Aug 25 '18 at 01:33

5 Answers5

27

No. Python always looks up special methods through the object's class. There are several good reasons for this, one being that repr(A) should use type(A).__repr__ instead of A.__repr__, which is intended to handle instances of A instead of the A class itself.

If you want different instances of A to compute their len differently, consider having __len__ delegate to another method:

class A(object):
    def __len__(self):
        return self._len()

a = A()
a._len = lambda: 2
user2357112
  • 260,549
  • 28
  • 431
  • 505
  • 2
    Good answer, takes into account OPs requirement of having different instances use a different method of determining len. Though why that is a requirement is beyond me... – jesseops Apr 11 '16 at 19:10
  • 5
    Any method whose name begins and ends with `__` would require this treatment. Because of the double underscores, these are sometimes called "dunder" methods. "Dirty deeds and their dunder chief!" – kindall Apr 11 '16 at 19:14
10

Special methods such as __len__ (double-underscore or "dunder" methods) must be defined on the class. They won't work if only defined on the instance.

It is possible to define non-dunder methods on an instance. However, you must convert your function to an instance method by adding a wrapper to it, which is how self gets passed in. (This would normally be done when accessing the method, as a method defined on the class is a descriptor that returns a wrapper.) This can be done as follows:

a.len = (lambda self: 2).__get__(a, type(a))

Combining these ideas, we can write a __len__() on the class that delegates to a len() that we can define on the instance:

class A(object):
     def __len__(self):
         return self.len()

a = A()
a.len = (lambda self: 2).__get__(a, type(a))

print(len(a))  # prints 2

You can actually simplify this in your case because you don't need self in order to return your constant 2. So you can just assign a.len = lambda: 2. However, if you need self, then you need to make the method wrapper.

kindall
  • 178,883
  • 35
  • 278
  • 309
  • Can you explain a bit more regarding the wrapper? @user2357112 answer seems to give me access to `self` of the instance even without the wrapper. – ARF Apr 11 '16 at 19:20
  • @ARF: No, I just left that out because your example `lambda` doesn't actually try to access `self`. If you do need `self`, you'd need to either get at it a different way (perhaps with a closure variable or `functools.partial`), or you'd need to do the `__get__` thing. – user2357112 Apr 11 '16 at 19:22
  • @user2357112 Thanks, after some playing around with it, I now understand the wrapper issue. – ARF Apr 11 '16 at 19:29
6

It is actually possible without modifying the class based on this answer by Alex Martelli:

class A(object):
    def __init__(self, words):
        self.words = words

    def get_words(self):
        return self.words


a = A("I am a")
b = A("I am b")

def make_meth(inst, _cls, meth, lm):
    inst.__class__ = type(_cls.__name__, (_cls,), {meth: lm})

make_meth(a, A, "__len__", lambda self: 12)
make_meth(b, A, "__len__", lambda self: 44)

print(len(b))
print(len(a))
print(a.get_words())
print(b.get_words())

If we run the code:

In [15]: a = A("I am a")    
In [16]: b = A("I am b")    
In [17]: make_meth(a, A, "__len__", lambda self: 12)
In [18]: make_meth(b, A, "__len__", lambda self: 44) 
In [19]: print(len(b))
44    
In [20]: print(len(a))
12

In [21]: print(a.get_words())
I am a    
In [22]: print(b.get_words())
I an b

As per the last part of the last part of the linked answer, you can add any methods on a per instance basis using inst.specialmethod once you have used inst.__class__ = type(... :

In [34]: b.__class__.__str__ =  lambda self: "In b"

In [35]: print(str(a))
<__main__.A object at 0x7f37390ae128>

In [36]: print(str(b))
In b
Community
  • 1
  • 1
Padraic Cunningham
  • 176,452
  • 29
  • 245
  • 321
  • Reassigning the object's class? I suppose it should work, as long as instances of A have a `__dict__` (otherwise you get a TypeError). It has a few other implications likely to be confusing to anyone who has to deal with an object modified this way, but perhaps not substantially more confusing than changing the behavior of `__len__` on someone else's class already is. – user2357112 Apr 11 '16 at 19:50
  • @user2357112, yes , it works without changing the class which was a requirement of the OP's that they did not add to the question, I did not say it was the best solution but it shows that it is possible. – Padraic Cunningham Apr 11 '16 at 19:57
  • @ARF, it should be `print(a.get_words())` with an s. I edited the answer and forgot to add the s – Padraic Cunningham Apr 11 '16 at 20:46
  • Am I understanding this right: this is basically a dynamic/implicit form of the wrapper-class solution @user2357112 has suggested? (Only that the wrapper class is created after instantiation of the object rather than before.) – ARF Apr 11 '16 at 20:49
  • It's not clear why you take `_cls` as a parameter here instead of using `inst.__class__`, and `_cls.__class__.__name__` is the metaclass name instead of the class name (which is why ARF's error message was saying `'type' object` instead of `'A' object`). – user2357112 Apr 11 '16 at 20:56
  • @ARF, no worries, it was my mistake. – Padraic Cunningham Apr 11 '16 at 20:58
  • @user2357112, because it was an oversight. I take _cls as the third argument is the class and it keeps the logic the same as Alex Martelli's example. Can you expand on *perhaps not substantially more confusing than changing the behaviour of __len__ on someone else's class already is*? The question is about changing the behaviour or adding if it does not exist on a per instance basis which is what is happening, any other instances are not affected so not sure I understand what you are getting at unless I am missing something, nothing else should be affected? – Padraic Cunningham Apr 11 '16 at 21:01
  • @ARF and Padraic: The "other implications" are a variety of subtle headaches that may or may not come up. For example, if `A` inherits `object.__repr__`, then objects modified this way will show up as `` instead of ``. Anyone testing the `type` of these objects rather than `isinstance` (possibly for good reason, e.g. optimizations that shouldn't be applied to subclass instances) could be surprised, though probably not any more surprised than they would be by getting weird `len` results. – user2357112 Apr 11 '16 at 21:12
  • @PadraicCunningham: Oh, are you asking about the "someone else's class" bit? I'm not saying that there are class-wide changes involved. It's just saying that if these objects are instances of a class you don't control (as they probably are if you can't change the `__len__` implementation), then changing how they respond to `len` is probably going to cause a lot of surprises, more than the "other implications" of reassigning the class. – user2357112 Apr 11 '16 at 21:19
  • @user2357112, yes, that was what I meant. I understand your points but I think this is a very particular use case, the len even if defined is going to be modified per instance so there is no real idea of a len method in the class, it may as well not be defined as whatever the class definition is will most likely not be match what will actually be happening. If you want a way to do it without changing the class definition then I don't imagine there are many other alternatives in the same style whether pretty or not. – Padraic Cunningham Apr 11 '16 at 21:25
  • @ARF, as far as anything going wrong I don't see much issues bar the slots issue already mentioned but if that were the case then `a.anything = ...` would have failed with an `AttributeError`. – Padraic Cunningham Apr 11 '16 at 23:02
  • There are a bunch more "subtle headaches" as yet unenumerated here. For example, type(a) != A, type(b) != A, type(a) != type(b). Also, all three classes have separate class dicts; assignments to A will be visible in a and b unless they occlude it (A.foo = 3; type(b).foo = 5; hasattr(b, "foo") == False). You could fix that by hand, type(a).__dict__ = A.__dict__ etc. – Larry Hastings Apr 16 '16 at 17:15
  • `make_meth`: are you Walter White ? – Jean-François Fabre Mar 30 '18 at 21:00
5

You have to define special methods at definition time of the class:

class A(object):
    def __len__(self):
        return self.get_len()

a = A()
a.get_len = lambda: 2
print len(a)
Daniel
  • 42,087
  • 4
  • 55
  • 81
1

You did not specify whether you're using Python 2.x or 3.x, and in this case it depends! In 3.x, your question has already been answered by other people. In 2.x, your question actually has an easy answer: stop deriving from object! (Of course, if you follow this approach, you can't upgrade to 3.x until you find another way to do this, so don't do what I'm suggesting unless you have no plans to upgrade any time soon.)

Type "copyright", "credits" or "license()" for more information.
>>> class A:pass

>>> a=A()
>>> a.__len__ = lambda: 2
>>> len(a)
2
>>> 

Enjoy :)

Mark VY
  • 1,489
  • 16
  • 31