Class variables and inheritance in Python

Question

I'd like to be able to define a class variable (in the base class somehow) that is present but not shared among the subclasses or instances. What I initially tried was the following:

from __future__ import print_function  # For lambda print()

class CallList(list):
    def __call__(self, *args, **kwargs):
        for f in self:
            f(*args, **kwargs)

class State:    
    on_enter = CallList()
    def enter(self):
        self.on_enter()

class Opening(State): pass 
class Closing(State): pass

Opening.on_enter.append(lambda: print('Opening state entered'))
Closing.on_enter.append(lambda: print('Closing state entered'))

but the behaviour of the subclasses and subclass instances is to reference the base classes class variable, which gives me the following:

opening = Opening()
closing = Closing()

opening.enter()
# Actual output:  Opening state entered
#                 Closing state entered
# Desired output: Opening state entered

opening.on_enter.append(lambda: print('Additional instance callback'))
opening.enter()
# Actual output:  Opening state entered
#                 Closing state entered
#                 Additional instance callback
# Desired output: Opening state entered
#                 Additional instance callback

closing.enter()
# Actual output:  Opening state entered
#                 Closing state entered
#                 Additional instance callback
# Desired output: Closing state entered

I understand why this is happening (I was expecting something different from experience is other languages, but that's fine).

Is is possible to modify the Base class (and not the subclasses) to get the subclasses to each have their own copy of the class variable, and for the instances of the subclasses to get a copy of their classes variable that can then independently be modified with no sharing as well?

I don't understand why Opening and Closing are subclasses, and why on_enter needs to be a class variable. If on_enter was an instance variable and Opening and Closing were instances of State, everything would work fine. — RemcoGerlich, Jul 22 '15 at 09:34
You can probably write a metaclass for the base class that ensures that subclasses don't "inherit" certain variables. — , Jul 22 '15 at 09:36
You can make each *class* have a separate class attribute with a metaclass (see e.g. http://stackoverflow.com/q/100003/3001761), but if each *instance* should have a separate attribute why not make it an instance attribute? — jonrsharpe, Jul 22 '15 at 09:41
So I want each instance to have the standard behaviours for that state (perhaps I have three actions I perform on entering all Opening states), so I was thinking that having them set at the class level was the best way to achieve that. I also want to be able to add additional behaviours to the instance (if this and that is true, also do this additional action on entering *this* instance of an Opening state). — Edouard Poor, Jul 22 '15 at 09:59

score 1 · Accepted Answer · answered Jul 22 '15 at 09:49

First of all, start using new-style classes by inheriting from object:

class State(object):    
    on_enter = CallList()
    def enter(self):
        self.on_enter()

Old-style classes are old, and what I’m going to suggest won’t work there.

Your specific problem can be solved with descriptors. As you can see in the documentation, descriptors allow us to override attribute access for a specific attribute, namely the attribute they are applied to. This is even possible when the attribute is read from the class.

import weakref

class CallListDescriptor(object):
    def __init__(self):
        self._class_level_values = weakref.WeakKeyDictionary()

    def __get__(self, instance, type):
        if instance is None:
            # class-level access
            try:
                return self._class_level_values[type]
            except KeyError:
                default = CallList()
                self._class_level_values[type] = default
                return default
        # instance-level access
        return instance._call_lists.setdefault(self, CallList())


class State(object):
    def __init__(self):
        # for the instance-level values
        self._call_lists = {}

    on_enter = CallListDescriptor()

We are using weakref for the class-level attributes to ensure that subclasses can get garbage collected properly while the superclass is still in scope.

We can test that it works:

class SubState(State):
    pass

class OtherSubState(State):
    pass

assert State.on_enter is State.on_enter
assert State.on_enter is not SubState.on_enter
assert State.on_enter is not OtherSubState.on_enter
assert SubState.on_enter is SubState.on_enter

instance = SubState()
assert instance.on_enter is not SubState.on_enter

I would however recommend to get rid of the subclass feature and merely ensure instance separate values and then represent state as instances of State instead of subclasses (except if you have a good reason not to, which there perfectly might be):

class CallListDescriptor(object):
    def __get__(self, instance, type):
        if instance is None:
            return self

        return instance._call_lists.setdefault(self, CallList())

class State(object):
    def __init__(self):
        # for the instance-level values
        self._call_lists = {}

That's looking very nice - the one thing it doesn't do, though, is have an instance of Closing inherit the actions I've assigned to the Closing class. I was wanting (in my initial example) for `opening.enter()` to print "Opening state entered" because I'd attached that action to the Opening Class. — Edouard Poor, Jul 22 '15 at 10:19
As for the sub-classes of class State design: this is a simplified example of something I'm working on - it's an experiment, but I'm getting good milage out of the subclasses/instances approach so far, so I think I'll stick with it for now. — Edouard Poor, Jul 22 '15 at 10:52
I did not read the requirement for inheritance from the question, this is another block of work. You could probably make use of the [``__mro__``](https://docs.python.org/2/library/stdtypes.html#class.__mro__) (which contains all base classes in their member lookup order) attribute of ``type`` inside ``__get__``. Not sure if you can figure it out from here, if not, I suggest you make a new question. Off the top of my head I’m not sure if this can be solved without a specialized metaclass though, as it might require knowing the name of the attribute. — Jonas Schäfer, Jul 22 '15 at 11:35
I hacked with the answer you gave, and I think I got something working.I'll mark your answer as correct, as it got me what I needed - thank-you. I'll tidy my code and post a gist here later. — Edouard Poor, Jul 23 '15 at 01:16
@EdouardPoor If you want feedback on your code, check out [codereview.stackexchange.com](https://codereview.stackexchange.com). — Jonas Schäfer, Jul 23 '15 at 09:13

Class variables and inheritance in Python

1 Answers1