Overriding tuple.len in python

Question

I would like to implement an immutable type in python natively. I gathered from here

How to make an immutable object in Python?

that the best way is to override tuple. Let's say I want a tuple that stores some extra data. I would do something like:

class MyTuple(tuple):
    def __new__(cls, lst, data):
        return tuple.__new__(cls, tuple(lst) + (data, ))

Now I would like to override the len function:

    def __len__(self):
        return tuple.__len__(self) - 1

This worked fine with my python version at home (Python 2.something, but I don't know which), but on my office computer (Python 2.7.3) it breaks slicing: If I do

m = MyTuple([1,2], 0)
print(m[:-1])

I get (1, ) while at home I would get (1, 2) (I didn't actually check, but tracing my problem back to this minimal example, I think so). So it seems in one implementation, slicing is calling tuple.__len__ while in the other it is calling MyTuple.__len__. I wouldn't mind either way, but consistency would be good.

So my question is: is there a robust solution and if not, which of the behaviors is going to be the stable one?

Edit: as Leonardo.Z suggested, my "home" python was actually Python 3(.4). I had been trying a new ide and not paid attention to which Python it used.

I am sure the Python version in your home is 3, not 2! – Leonardo.Z May 15 '14 at 10:07 — Leonardo.Z, May 15 '14 at 10:07

score 1 · Answer 1 · answered May 15 '14 at 09:10

1

Slicing does not necessarily relate to calling __len__(), especially not with built-in types.

m[:-1] translates to m.__getitem__(slice(None, -1)), which might be handled in the tuple type in a very easy way by calling an internal length function which iggnores the fact that __len__ is overridden.

Here is how the function is internally defined.

It seems that it makes extensive use of PyTuple_GET_SIZE(). I didn't find its definition, but I suppose it is defined quite simply.

answered May 15 '14 at 09:10

glglgl

89,107
13
149
217

`PyTuple_GET_SIZE` is defined in [`tupleobject.h`](http://hg.python.org/cpython/file/0a6f0aaeb96a/Include/tupleobject.h) as an alias for `Py_SIZE`, which reads the standard `ob_size` entry in the [`PyObject_VAR_HEAD` struct](http://hg.python.org/cpython/file/0a6f0aaeb96a/Include/object.h#l90). – Martijn Pieters May 15 '14 at 09:43
@MartijnPieters Thx, now I found it, and so it does, so it indeed bypasses calling `__len__()` altogether. – glglgl May 19 '14 at 08:33

Leonardo.Z · Accepted Answer · 2014-05-15T10:02:04.507

It is caused by a difference between the slice implementation of Python2 and Python3.

In Python3, obj[x:y] always equals obj.__getitem__(slice(x,y)). But in Python2, due to historical reasons, some built-in types(including tuple) implement the slice operator with the __getslice__ method.

__getslice__ uses the __len__ method, __getitem__ does not.

Just run the following code in Python2 and Python3, and you'll see why.

from __future__ import print_function

class MyTuple(tuple):
    def __new__(cls, lst, data):
        return tuple.__new__(cls, tuple(lst) + (data, ))
    def __len__(self):
        return tuple.__len__(self) - 1

    def __getitem__(self, key):
        print("__getitem__() is called.  start:%s, end:%s" % (key.start, key.stop))
        return super(MyTuple, self).__getitem__(key)

    def __getslice__(self, start, end):
        print("__getslice__() is called.  start:%s, end:%s" % (start,end))
        return super(MyTuple, self).__getslice__(start,end)

m = MyTuple([1,2], 0)

print(m[:-1])

According to the Python official doc, you should override __getslice__() method when you change then __len__ implementation in your subclass. Reference:

The official doc about object.__getslice__.

Overriding tuple.__len__ in python

2 Answers2

Overriding tuple.len in python