2

The following code works perfectly in Python 2.x:

class a:
    def __init__ (self, *args):
        self.lst = list(args)
        self.__len__ = self.lst.__len__

b = a(1, 2, 3)
print(len(b))

In Python 3 it shouts:

TypeError: object of type 'a' has no len()

WHAT HAVE THEY DONE!!!!!!!!????

I disparately need this feature to work, and only time i succeeded was changing it in a class e.g.:

b = a()
b.__class__.__len__ = lambda: 123

But this changes the class, not my instance, and therefore it is not thread safe, and it is stupid besides. It does not work even inside new() instead of init() which I thought it might.

What is even worse, if I do as shown above (with the lambda) on an instance then:

>>> hasattr(b, "__len__")
True
>>> b.__len__()
123
>>> # But:
>>> len(b)
Traceback...
TypeError: object 'b' has no len()

What is alarming is that when I define the len() wrapper inside a class, then change it from outside or within an method, it changes, but len() somehow gets to the original method. I am guessing that this have something to do with trying to introduce statically referenced names at compile time instead of keeping all of them in a dict. But who cares, I need it to work.

clas c:
    def __len__ (self):
        return 123
>>> d = c()
>>> len(d)
123
>>> d.__len__ = lambda: 333
>>> len(d)
123
>>> d.__len__()
333

Changing d.__dict__ or using setattr()

doesn't help either. How, or rather, where, are the pointers kept apart? Interesting behaviour.

If I redefine the len() function:

len = lambda o: o.__len__()

It may save me for the len() problem, but what is with .__contains__, .__eq__, etc.

I need a custom container and I need to bind the underlying datastructure's wrapper methods to my instances for micro optimization. Besides, I have codes with a lot of such tricks and I need to rewrite them for Py3. I do not plan to spend the eternity hunting for these lines in a code and adding methods that unnecessarily call other methods. This is bad.

What do I do?

izhang05
  • 744
  • 2
  • 11
  • 26
Dalen
  • 4,128
  • 1
  • 17
  • 35
  • 3
    "But this changes the class, not my instance, and therefore it is not thread safe" Why can't you just have the class have the method in the first place? I don't understand the problem you are trying to solve, and I don't understand how this is expected to optimize anything it did work. Can you show evidence in the form of benchmarks run under 2.x on your machine? – Karl Knechtel Apr 18 '20 at 02:18
  • @KarlKnechtel There are no benchmarks needed. Bound functions are faster firstly because you do not go one extra scope down to fetch them. It is self.__len__, not self.lst.__len__, then, you have the len() which calls the obj.__len__(), if I create my __len__() then len(), calls obj.__len__() which calls obj.lst.__len__(). So you have to create stack for each called function, locals etc., then find the function/method it calls, which has to do the same, then you return back.... blaaah... I am doing DSP on a sound which is synthesized and must be played in realtime. No delays tolerated. – Dalen Apr 18 '20 at 02:50
  • 2
    dunder methods part of the official data model are looked up *first on the type* now in Python 3, skipping the instance. Your optimization wouldn't matter, since it's already been optimized. – juanpa.arrivillaga Apr 18 '20 at 02:57
  • Note, this only would have worked on old-style classes in Python 2 anyway, something you should have migrated away from ages ago... – juanpa.arrivillaga Apr 18 '20 at 03:00
  • @juanpa.arrivillaga BTW, I see the wisdom of new style classes implementation, but why then in Py3 len() accesses obj.__class__.__len__() while you are permitted to define a new __len__() which then doesn't do what it is supposed to do. Some things like removing mixing of bytes() and str() just because str() is unicode() doesn't make sense to me, and at the same time, you can have two different __len__()s no problem. This behaviour is a little bit inconsistent. – Dalen Apr 18 '20 at 03:41
  • It accesses the class because normal code writes the method in the class and it's faster that way. You can define a (comparatively) useless method attached to the instance because Python is not in the habit of prohibiting things it doesn't need to prohibit. The changes to `bytes` and `str` have strong technical reasons behind them that are beyond the scope of this post; and if the point of all of this is really just to complain about 3.x then it is definitely off topic for SO. – Karl Knechtel Apr 18 '20 at 03:47
  • There is no way around this in Python 3. If you rely on dirty hacks, this is sort of the situation you find yourself in when changes are made. Note, it accesses `type(instance).__len__` for special methods in the context of the hooks that they are... In any case, I don't think there is any good way to do this for you, especially if you must make the code 2-3 compatible. – juanpa.arrivillaga Apr 18 '20 at 03:58
  • @juanpa.arrivillaga Thank you for making me laugh. Anyway, I can't. My program is a plugin of another program that is run by CPython. And it basically changes the interpreter through its versions as soon as another pops up. There was a little lag when 3 came, but now, well, you know the situation. So the program in question runs from 2.6 to 3.7. – Dalen Apr 18 '20 at 04:23
  • What is even worse, my program has its own plugin system, and, well a debugger for its plugins. And I just discovered that inspect.formatargspec() radically changed. I wasn't expecting such a mess. I am doubley screwed. – Dalen Apr 18 '20 at 04:27
  • Yeah, I'm sorry. That sounds like quite mess. This will be quite the war story. – juanpa.arrivillaga Apr 18 '20 at 04:32
  • Hey, @juanpa.arrivillaga does it really do type(obj).__*__() ??? Because if it does call the type() from our scope, not the C implementation of it, then we can overwrite it with our version that would return the instances wrapper method, not the class defined one. Of course, optimization goes to hell, but at least code works. And I do not call such things dirty hacks, but clever ways to fully utilize all the Python's power and might. – Dalen Apr 18 '20 at 15:44

0 Answers0