10

From https://stackoverflow.com/a/1529099/156458

To support arbitrary attribute assignment, an object needs a __dict__: a dict associated with the object, where arbitrary attributes can be stored. Otherwise, there's nowhere to put new attributes.

An instance of object does not carry around a __dict__ -- if it did, before the horrible circular dependence problem (since __dict__, like most everything else, inherits from object;-), this would saddle every object in Python with a dict, which would mean an overhead of many bytes per object that currently doesn't have or need a dict (essentially, all objects that don't have arbitrarily assignable attributes don't have or need a dict).

...

When the class has the __slots__ special attribute (a sequence of strings), then the class statement (more precisely, the default metaclass, type) does not equip every instance of that class with a __dict__ (and therefore the ability to have arbitrary attributes), just a finite, rigid set of "slots" (basically places which can each hold one reference to some object) with the given names.

If an object doesn't have __dict__, must its class have a __slots__ attribute?

For example, an instance of object doesn't have a __dict__:

>>> object().__dict__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'object' object has no attribute '__dict__'

but it doesn't have __slots__ either:

>>> object.__slots__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'object' has no attribute '__slots__'

Do object instances have any attributes at all?

How many possibilities are there:

  • an object has __dict__, and its class has __dict__ but no __slots__
  • an object doesn't have __dict__, and its class has __slots__
  • an object doesn't have __dict__, and its class doesn't have __slots__ ?

Is it possible to tell if an object has __dict__ from its class?

  • if its class has __slots__, then it doesn't have __dict__, correct?

  • if its class doesn't have __slots__, how can I tell if it has __dict__ or not?

Karl Knechtel
  • 62,466
  • 11
  • 102
  • 153
Tim
  • 1
  • 141
  • 372
  • 590

1 Answers1

15

For user defined classes (defined using the class keyword in regular Python code), a class will always have __slots__ on the class, __dict__ on the instance, or both (if one of the slots defined is '__dict__', or one of the user defined classes in an inheritance chain defines __slots__ and another one does not, creating __dict__ implicitly). So that's three of four possibilities covered for user defined classes.

Edit: A correction: Technically, a user-defined class could have neither; the class would be defined with __slots__, but have it deleted after definition time (the machinery that sets up the type doesn't require __slots__ to persist after the class definition finishes). No sane person should do this, and it could have undesirable side-effects (full behavior untested), but it's possible.

For built-in types, at least in the CPython reference interpreter, they're extremely unlikely to have __slots__ (if they did, it would be to simulate a user-defined class, defining it doesn't actually do anything useful). A built-in type typically stores its attributes as raw C level values and pointers on a C level struct, optionally with explicitly created descriptors or accessor methods, which eliminates the purpose of __slots__, which are just a convenient limited purpose equivalent of such struct games for user defined classes. __dict__ is opt-in for built-in types, not on by default (though the opt-in process is fairly easy; you need to put a PyObject* entry somewhere in the struct and provide the offset to it in the type definition).

To be clear, __dict__ need not appear on the class for it to appear on its instances; __slots__ is class level, and can suppress the __dict__ on the instance, but has no effect on whether the class itself has a __dict__; user defined classes always have __dict__, but their instances won't if you're careful to use __slots__ appropriately.

So in short:

(Sane) User defined classes have at least one of __dict__ (on the instances) or __slots__ (on the class), and can have both. Insane user defined classes could have neither, but only a deranged developer would do it.

Built-in classes often have neither, may provide __dict__, and almost never provide __slots__ as it is pointless for them.

Examples:

# Class has __slots__, instances don't have __dict__
class DictLess:
    __slots__ = ()

# Instances have __dict__, class lacks __slots__
class DictOnly:
    pass

# Class has __slots__, instances have __dict__ because __slots__ declares it
class SlottedDict:
    __slots__ = '__dict__',

# Class has __slots__ without __dict__ slot, instances have it anyway from unslotted parent
class DictFromParent(DictOnly):
    __slots__ = ()

# Complete insanity: __slots__ takes effect at class definition time, but can
# be deleted later, without changing the class behavior:
class NoSlotNoDict:
    __slots__ = ()
del NoSlotNoDict.__slots__
# Instances have no __dict__, class has no __slots__ but acts like it does
# (the machinery to make it slotted isn't undone by deleting __slots__)
# Please, please don't actually do this

# Built-in type without instance __dict__ or class defined __slots__:
int().__dict__  # Raises AttributeError
int.__slots__   # Also raises AttributeError

# Built-in type that opts in to __dict__ on instances:
import functools
functools.partial(int).__dict__ # Works fine
functools.partial.__slots__ # Raises AttributeError
ShadowRanger
  • 143,180
  • 12
  • 188
  • 271
  • Thanks. Is it possible to tell if an object has `__dict__` merely from its class? If its class has `__slots__`, then it doesn't have `__dict__`. If its class doesn't have `__slots__`, how can I tell if it has `__dict__` or not? – Tim Oct 04 '17 at 22:53
  • @Tim: `hasattr(obj, '__dict__')`? – Kevin Oct 04 '17 at 22:56
  • @Tim: A class that has `__slots__` can still have `__dict__` (if it includes `'__dict__'` as a slot name, or a parent class didn't define `__slots__`). If the class doesn't have `__slots__`, it could be a built-in type without `__dict__`, or a user defined class with a `__dict__`. Make an instance, check it for `__dict__`, that's the reliable way. – ShadowRanger Oct 04 '17 at 22:56
  • by "A class that has `__slots__` can still have `__dict__`", do you mean "if a class has `__slots__`, its instances can still have `__dict__`" or "if a class has `__slots__`, the class itself can still have `__dict__`" ? – Tim Oct 04 '17 at 23:00
  • 1
    @Tim: User defined classes themselves *always* have a `__dict__`; you can suppress it on the instances with `__slots__`. – ShadowRanger Oct 04 '17 at 23:05
  • Note: the closest you can come to "is this a python or a C type" is testing `__flags__ & (1 << 9)` – o11c Oct 04 '17 at 23:07
  • @o11c: And I'm not sure that's a documented guarantee even for CPython, let alone the various alternate interpreters. – ShadowRanger Oct 04 '17 at 23:18
  • Thanks. (1) ["To support arbitrary attribute assignment, an object needs a `__dict__`"](https://stackoverflow.com/a/1529099). So is it correct that `__dict__` as an attribute of a class or instance is used for storing user defined attributes of the class or instance? (2) What do you mean by "Built-in classes often have neither, may provide `__dict__`" and "`__dict__` is opt-in for built-in types"? Is `__dict__` for a builtin class used for storing user-defined attributes? – Tim Oct 05 '17 at 01:34
  • To #1: Yes, unless the attributes are slotted, in which case they're not stored in `__dict__`. To #2. Exactly what I said. Built-ins can, but almost never do, use `__dict__` to store instance attributes. When they provide `__dict__`, it's usually to allow (for whatever reason) the user to add additional arbitrary attributes to instances of the class. – ShadowRanger Oct 05 '17 at 01:39
  • For example, `functools.partial` didn't have to provide `__dict__`, and doesn't use it itself (it [stores all the stuff it actually uses in top-level pointers in the C struct](https://github.com/python/cpython/blob/3.6/Modules/_functoolsmodule.c#L14)), but it provides it (and makes sure it survives pickling and unpickling), probably because regular functions allow you to set arbitrary additional attributes, and they wanted it to behave like a normal function object (since it wraps functions and otherwise behaves like one). – ShadowRanger Oct 05 '17 at 01:41
  • Thanks. `object` is a builtin class, but `object.__dict__` is not empty by default,. `__dict__` is meant to store user-defined attributes, while `object.__dict__` stores builtin attributes of `object`? – Tim Oct 05 '17 at 02:07
  • Nothing mandates that *only* user defined stuff go in `__dict__`. Classes have `__dict__`, always, but if you actually look at the `__dict__` for `object`, it's mostly just a bunch of slot wrappers; descriptors that provide a Python-level names for the C level function pointers associated with the type (which are dynamically defined, because not all slots are populated on each type). In any event, you're trying to draw hard & fast lines that don't exist. `__dict__` is a simple, convenient way to store arbitrary attributes. Nothing *requires* that it only be used for user specified attributes. – ShadowRanger Oct 05 '17 at 03:47
  • As it happens, it looks like `type` subclasses (which are what classes themselves are instances of) *can't* define non-empty `__slots__` (and non-empty `__slots__` doesn't actually block `__dict__`), possibly due to optimization related concerns for super class lookups and the like, possibly because trying to do so would interfere with the automatically generated attributes Python creates for classes; for classes, the only place they have to store arbitrary data is `__dict__`, thus the observed behavior. – ShadowRanger Oct 05 '17 at 03:57
  • "no sane person should do this" well guess what they did it :P look at ClientUser in discord,py – John Sohn Sep 18 '21 at 13:29
  • @JohnSohn: I don't see anything obviously weird there. They define `__slots__` and never delete them. Only slightly weird thing they do is actually *use* `__slots__` programmatically in other contexts (to make a mix-in named `_RawReprMixin` that defines `__repr__` without naming all the attributes explicitly by relying on the `__slots__` of the class), and that's only lightly weird (it's a brittle hack that only works for the most derived subclass, ignoring the `__slots__` of the parent classes). – ShadowRanger Apr 18 '22 at 14:49
  • A user-defined class could also be created by directly using `type`. – Karl Knechtel May 24 '22 at 00:17