14

With the following code :

import types

class Foo():
    def __getitem__(self, x):
        return x

def new_get(self, x):
    return x + 1

x = Foo()
x.__getitem__ = types.MethodType(new_get, x)

x.__getitem__(42) will return 43, but x[42] will return 42.

Is there a way to override __getitem__ at instance level in Python?

ruohola
  • 21,987
  • 6
  • 62
  • 97
Statistic Dean
  • 4,861
  • 7
  • 22
  • 46
  • I'm sure there is – WiseDev Aug 08 '19 at 13:16
  • 2
    You wouldn't want to do this anyway. An object with a different method definition isn't, conceptually, part of the class anymore. – chepner Aug 08 '19 at 13:18
  • If you want the full story: I want to time some process, so basically, I override methods of the instance I want to time by adding code to time the call to those methods, in particular, I want to do that for call to `__getitem__`. (The idea is to modify an object, pass it through some existing code and record the time it took at each call of a method). – Statistic Dean Aug 08 '19 at 13:23
  • 1
    Related: [“implicit uses of special methods always rely on the class-level binding of the special method”](https://stackoverflow.com/q/46054208/364696) (the text is wrong actually; not all special methods rely on the class-level binding, but you can't count on any given one of them not doing so, and it could change from version to version because instance level rebinding is not officially supported for any of them). – ShadowRanger Aug 08 '19 at 14:39
  • this is annoying. I want to do it anyway. Is it possible to do it anyway in python3? – Charlie Parker Mar 16 '23 at 01:20

3 Answers3

11

This is unfortunately, and quite surprisingly, not allowed:

For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary.

Source: https://docs.python.org/3/reference/datamodel.html#special-lookup

ruohola
  • 21,987
  • 6
  • 62
  • 97
  • 1
    Searched on the docs a little and couldn't find this info. Thanks a lot ! I guess i'll have to find another solution to achieve my desired solution. – Statistic Dean Aug 08 '19 at 13:21
  • 1
    An attempt at this: `x.__class__.__bases__.__getitem__ = new_get` gives `AttributeError: 'tuple' object attribute '__getitem__' is read-only`, so you're right. – WiseDev Aug 08 '19 at 13:22
  • It's not exactly unfortunate; if they allowed it, you'd be able to, for example, subscript the sequence classes themselves, e.g. `list[2]`, and rather than immediately rejecting that as nonsensical, it would have to have a check in `list.__getitem__` to make sure it was passed a `list` instance, not the class itself, *every* time. Looking it up on the class only also speeds things up in other ways (99.9% of the time it would come from C level slots on the class, but you'd have to check the instance dictionary *every* time to make sure it wasn't the < 0.1% case where it was overridden). – ShadowRanger Aug 08 '19 at 14:37
  • @ShadowRanger Yeah, just meant unfortunately for OP's question. – ruohola Aug 08 '19 at 14:53
  • this is annoying. I want to do it anyway. Is it possible to do it anyway in python3? – Charlie Parker Mar 16 '23 at 01:20
4

Don't do it...

The item lookup protocol will always recover __getitem__ from the class, it will not even look at instance __dict__. This is actually a good thing in general as doing otherwise would allow instances of the same class to be conceptually different from one another, which goes against the whole idea behind classes.

But...

Nonetheless, there are situation where this could potentially be helpful, by example when monkey-patching for test purpose.

Because the dunder is looked up directly at class level, the item lookup logic must also be updated at the class level.

A solution is thus to update __getitem__ so that it first looks for an instance-level function in the instance __dict__.

Here is an example where we are subclassing dict to allow for instance-level __getitem__.

class Foo(dict):
    def __getitem__(self, item):
        if "instance_getitem" in self.__dict__:
            return self.instance_getitem(self, item)
        else:
            return super().__getitem__(item)

foo = Foo()
foo.instance_getitem = lambda self, item: item + 1
print(foo[1]) # 2
Ciantic
  • 6,064
  • 4
  • 54
  • 49
Olivier Melançon
  • 21,584
  • 4
  • 41
  • 73
  • 1
    This is a very good suggestion, but I don't think it will help me in my case. The idea is that I want to time some calls of `__getitem__` inside some code not written by me. So I take an object, modify it's `__getitem__` method to record time on each call, and just pass it to the code not written by me. I'll just have to do it on class level instead of instance level. – Statistic Dean Aug 08 '19 at 13:26
  • 1
    @StatisticDean The above snippet of code shows you how to do what you want with inheritance – Olivier Melançon Aug 08 '19 at 14:33
  • this is annoying. I want to do it anyway. Is it possible to do it anyway in python3? – Charlie Parker Mar 16 '23 at 01:21
  • @CharlieParker This solution works with Python3 if that is what you are asking. Although, you can also directly use a dict which contain functions which you can call explicitly. – Olivier Melançon Mar 16 '23 at 01:51
  • I just want `d.__getitem__ = new_getitem` to work. Your solution requires me to hard code per object e.g. `dict` the behaviour. I have multiple data set objects that need this as an input to modify the class. Basically, I don't want to write a new class. – Charlie Parker Mar 16 '23 at 17:48
1

I ended up having to do something stupid like this just making a new object, calls the old __getitem__ and does something different:

class USLDatasetFromL2L(datasets.Dataset):

    def __init__(self, original_l2l_dataset: datasets.Dataset):
        self.original_l2l_dataset = original_l2l_dataset
        self.transform = self.original_l2l_dataset.transform
        self.original_l2l_dataset.target_transform = label_to_long
        self.target_transform = self.original_l2l_dataset.target_transform

    def __getitem__(self, index: int) -> tuple:
        """ overwrite the getitem method for a l2l dataset. """
        # - get the item
        img, label = self.original_l2l_dataset[index]
        # - transform the item only if the transform does exist and its not a tensor already
        # img, label = self.original_l2l_dataset.x, self.original_l2l_dataset.y
        if self.transform and not isinstance(img, Tensor):
            img = self.transform(img)
        if self.target_transform and not isinstance(label, Tensor):
            label = self.target_transform(label)
        return img, label

    def __len__(self) -> int:
        """ Get the length. """
        return len(self.original_l2l_dataset)
Charlie Parker
  • 5,884
  • 57
  • 198
  • 323