2

In Python 2.7, I can declare a new-style class and set an attribute using object.__setattr__:

class A (object):
    def __init__ (self):
        object.__setattr__(self, 'foo', 'bar')

A().foo # 'bar'

However, this does not work in an old-style class:

class B:
    def __init__ (self):
        object.__setattr__(self, 'foo', 'bar')

B().foo

TypeError: can't apply this __setattr__ to instance object

I know that old-style classes do not support this behaviour, but I do not know why it doesn't work.

What is going on under the hood when I call object.__setattr__ on a new-style class that cannot occur for old-style classes?

Please note that I am not using this code for anything, I am just trying to understand it.

Ninjakannon
  • 3,751
  • 7
  • 53
  • 76
  • `object` is the base for new-style classes, so no, you can't use `object.__setattr__` on instances that are not inheriting from that base. – Martijn Pieters Jun 30 '17 at 20:25
  • @MartijnPieters I don't understand why that is - I am missing something here about how this works – Ninjakannon Jun 30 '17 at 20:26
  • *Please note that I am aware that there are only very special circumstances when one would use object.__setattr__ instead of setattr*. Do enlighten us, in what circumstances would you want to not use `setattr()` here? – Martijn Pieters Jun 30 '17 at 20:26
  • @MartijnPieters Please see [this question](https://stackoverflow.com/q/7559170/604687). You can also find [examples of this usage in the pandas core](https://github.com/pandas-dev/pandas/blob/f6f5ce5f9ce2afc3b6f55a3228b93024b121b88f/pandas/core/generic.py#L130). – Ninjakannon Jun 30 '17 at 20:27
  • Well, if you don't inherit from `object` that one case *doesn't apply*. The exception there being that an intermediary class in the MRO is being skipped. – Martijn Pieters Jun 30 '17 at 20:28
  • @MartijnPieters Ha, good point. Clearly this is somewhat contrived, but I didn't want people asking me in the comments why I was doing this as I'm not using it for anything - I'll re-word – Ninjakannon Jun 30 '17 at 20:39

2 Answers2

6

The object.__setattr__ method includes a specific check to make sure that self passed in is really a (subclass) of object (or another object that reuses the same C function directly). That test doesn't allow for anything else being passed in, including old-style instances.

The check is there to prevent object.__setattr__ being used to modify built-in types (known as the Carlo Verre hack, after its discoverer.

Without the explicit check (or if you compiled Python to disable it), you could do this:

>>> object.__setattr__(str, 'lower', str.upper)
>>> "This is why we can't have nice things".lower()
"THIS IS WHY WE CAN'T HAVE NICE THINGS"

See the hackcheck() function, and the original Python-dev discussion.

That old-style instance objects don't pass the test is just a side-effect.

Note that the only reason you'd ever want to call object.__setattr__ is if you have a base class that implements a __setattr__ method you need to bypass. For old-style instances, you could just use self.__dict__[name] = value instead.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • This is a great answer, thank you. I did not see any reason *why*, technically, this call would fail. Now I understand. – Ninjakannon Jun 30 '17 at 20:41
  • 1
    Of course, you can still do `gc.get_referents(str.__dict__)[0]['lower'] = str.upper` to make `str.lower` uppercase things anyway, so it's still pretty easy to screw with built-in types if you really want to. – user2357112 Jun 30 '17 at 20:54
  • @user2357112: yup, there are other loopholes to exploit, and perhaps those too should be closed. Wanna bring that one to the dev list and see if it is worth closing? – Martijn Pieters Jun 30 '17 at 21:00
  • 1
    @user2357112: actually, that one doesn't work for me, not in 2.7 at least. In 3.6 it leads to a segfault! – Martijn Pieters Jun 30 '17 at 21:01
  • @MartijnPieters: [Here's an Ideone demo](http://ideone.com/wpR1HB) you can compare against to see if you did something different from what I did. – user2357112 Jun 30 '17 at 21:04
  • @user2357112 I get segfaults 2.7 and 3.6 running that exact code in a python shell, but not in 2.7 when executing a script – Ninjakannon Jun 30 '17 at 21:06
  • Try it in a fresh interpreter. I think the `lower` lookup is hitting the method cache and trying to access the old `str.lower`, which has probably been deallocated. (This kind of thing is unsupported and dangerous, so it's not much of a surprise that crazy things go wrong.) – user2357112 Jun 30 '17 at 21:12
  • @user2357112: yup, it then works in a 2.7 interpreter session. 3.6 still segfaults (both in the REPL and as a script). – Martijn Pieters Jul 01 '17 at 09:30
  • @user2357112: I'm sure this can be done using ctypes too, I just can't be bothered to write that all out. – Martijn Pieters Jul 01 '17 at 09:34
1

The type - not the old-style class, but the type - of all old-style instances is types.InstanceType. This type is written in C and implements its own __setattr__. Python will refuse to apply the __setattr__ of one C type to instances of a C type with a different C-level __setattr__, as a safety measure to protect the consistency of C data structures.

user2357112
  • 260,549
  • 28
  • 431
  • 505
  • Yet the C data structures are the same here; with old-style attributes there are no descriptors that can hook into `__set__` anyway, so setting an attribute is always a `__dict__[attrname] = attrvalue` operation. – Martijn Pieters Jun 30 '17 at 20:41
  • @MartijnPieters: They're actually not quite the same - for example, InstanceType doesn't provide the metadata `object.__setattr__` would need to locate the instance's `__dict__` in the first place. – user2357112 Jun 30 '17 at 20:49
  • Right, indeed, one has `in_dict`, the other a `tp_dictoffset` offset into the slots. Still, could have been handled in an abstraction internally. – Martijn Pieters Jun 30 '17 at 20:59