correct way to use class-level primitives

Question

The way that class variables in Python are handled does not make any sense to me. It seems that the scope of a class variable is dependent upon its type! Primitive types are treated like instance variables, and complex types are treated like class variables:

>>> class A(object):
...   my_class_primitive = True
...   my_class_object = ['foo']
... 
>>> a = A()
>>> a.my_class_primitive, a.my_class_object
(True, ['foo'])
>>> b = A()
>>> b.my_class_primitive, b.my_class_object
(True, ['foo'])
>>> a.my_class_object.append('bar')
>>> b.my_class_primitive, b.my_class_object
(True, ['foo', 'bar'])
>>> a.my_class_primitive = False
>>> b.my_class_primitive, b.my_class_object
(True, ['foo', 'bar'])
>>> a.my_class_primitive, a.my_class_object
(False, ['foo', 'bar'])

Can someone please explain the following:

1. Why does this feature exist? What is the logic behind it?
2. If I want to use a primitive type (e.g. bool) as a class variable, how do I do it?

Answer 1

The feature exists as a form of caching for Python's class definitions.

Attributes defined in the class definition itself are considered static attributes of the class. e.g. they should not be modified.

I'm sure the decision was made assuming you were following best practices with respect to modifying static attributes ;)

Answer 2

The proper comparison you should be making is this:

>>> class A(object):
    my_class_primitive = True
    my_class_object = ['foo']


>>> a = A()
>>> b = A()
>>> a.my_class_primitive = False
>>> a.my_class_object = ['bar']
>>> b.my_class_primitive, b.my_class_object
(True, ['foo'])

This is not what you thought you see. The "strange" behavior is caused by the concept of mutability. Namely, you are changing object assigned to the my_class_object property, while you are assigning different object to my_class_primitive. In first case it works for all the instances (because this is one object), while in the second case it works for single instance (whose property you are changing).

You are not alone in that confusion. To see another example of how mutability influences your code, you can check this: "Least Astonishment" and the Mutable Default Argument

Answer 3

>>> class A(object):
...   my_class_primitive = True
...   my_class_object = ['foo']

Class attributes are stored in A.__dict__:

>>> A.__dict__
>>> <dictproxy {'__dict__': <attribute '__dict__' of 'A' objects>,
 '__doc__': None,
 '__module__': '__main__',
 '__weakref__': <attribute '__weakref__' of 'A' objects>,
 'my_class_object': ['foo'],
 'my_class_primitive': True}>

>>> a = A()
>>> a.my_class_primitive, a.my_class_object
(True, ['foo'])
>>> b = A()
>>> b.my_class_primitive, b.my_class_object
(True, ['foo'])

The statement a.my_class_object.append('bar') mutates the list A.my_class_object. It retrieves the existing list, a.my_class_object and then calls its append method:

>>> a.my_class_object.append('bar')

So b.my_class_object is affected too:

>>> b.my_class_primitive, b.my_class_object
(True, ['foo', 'bar'])

The assignment a.my_class_primitive = False add a new instance attribute to a. It places a key-value pair in a.__dict__:

>>> a.my_class_primitive = False

>>> a.__dict__
>>> {'my_class_primitive': False}

So b is not affected:

>>> b.my_class_primitive, b.my_class_object
(True, ['foo', 'bar'])

Python's attribute lookup rules cause a.my_class_primitive to return the value of the key found in a.__dict__ rather than the value of the key found in A.__dict__:

>>> a.my_class_primitive, a.my_class_object
(False, ['foo', 'bar'])

Answer 4

It's not about primitive and complex. When you append to a.my_class_object, you modify the existing object. When you assign to a variable, you don't modify it. The same "problem" exists for lists if you treat them like you did the boolean:

>>> class Foo(object):
...     x = []
...
>>> i1 = Foo()
>>> i2 = Foo()
>>>
>>> i1.x = 5
>>>
>>> print(i2.x)
[]

When you get i1.x, Python looks at i1.__dict__ for the attribute. If it can't find it in there, it looks in the __dict__ of each of that object's parent classes until it does (or throws an AttributeError). The returned object doesn't have to be an attribute of i1 at all.

When you assign to i1.x, you specifically assign to i1.x.

To modify a class attribute, refer to the class, not to an instance:

>>> class Foo(object):
...     x = 2
...
>>> i1 = Foo()
>>>
>>> Foo.x = 5
>>>
>>> print(i1.x)
5

Answer 5

The difference is not between primitives and complex types, the difference is between assignment and modifying a variable.

If you use instance.name = value you will always be assigning a new instance variable, even if a class attribute of the same name already exists.

For example:

>>> class A(object):
...   my_class_primitive = True
...   my_class_object = ['foo']
...
>>> a = A()
>>> a.my_class_primitive = False
>>> a.__class__.my_class_primitive
True
>>> a.__dict__
{'my_class_primitive': False}

The difference in behavior here with appending to a list is because when you do an attribute lookup on a class instance it will first look for instance variables and if there is not an instance variable with that name it will look for a class attribute. For example:

>>> a.my_class_object is a.__class__.my_class_object
True

So when you do a.my_class_object.append(x) you are modifying the class attribute which any instance can access. If you were to do a.my_class_object = x then it would not modify the class attribute in any way, it would just create a new instance variable.