Python : metaclass + wrapped methods + inheritance = problems

Question

I have a problem in Python, for which I cannot find any clean solution ...

When calling some methods, I want to execute some code before the method execution and after. In order (among many other things) to automatically set and clean a context variable.

In order to achieve this, I have declared the following metaclass :

class MyType(type):
    def __new__(cls, name, bases, attrs):
        #wraps the 'test' method to automate context management and other stuff
        attrs['test'] = cls.other_wrapper(attrs['test'])
        attrs['test'] = cls.context_wrapper(attrs['test'])
        return super(MyType, cls).__new__(cls, name, bases, attrs)

    @classmethod
    def context_wrapper(cls, operation):
        def _manage_context(self, *args, **kwargs):
            #Sets the context to 'blabla' before the execution
            self.context = 'blabla'
            returned = operation(self, *args, **kwargs)
            #Cleans the context after execution
            self.context = None
            return returned
        return _manage_context

    @classmethod
    def other_wrapper(cls, operation):
        def _wrapped(self, *args, **kwargs):
            #DO something with self and *args and **kwargs
            return operation(self, *args, **kwargs)
        return _wrapped

This works like a charm :

class Parent(object):

    __metaclass__ = MyType

    def test(self):
        #Here the context is set:
        print self.context #prints blabla

But as soon as I want to subclass Parent, problems appear, when I call the parent method with super :

class Child(Parent):
    def test(self):
        #Here the context is set too
        print self.context #prints blabla
        super(Child, self).test()
        #But now the context is unset, because Parent.test is also wrapped by _manage_context
        #so this prints 'None', which is not what was expected
        print self.context

I have thought of saving the context before setting it to a new value, but that only solves partially the problem...

Indeed, (hang on, this is hard to explain), the parent method is called, the wrappers are executed, but they receive *args and **kwargs addressed to Parent.test, while self is a Child instance, so self attributes have irrelevant values if I want to challenge them with *args and **kwargs (for example for automated validation purpose), example :

@classmethod
def validation_wrapper(cls, operation):
    def _wrapped(self, *args, **kwargs):
        #Validate the value of a kwarg
        #But if this is executed because we called super(Child, self).test(...
        #`self.some_minimum` will be `Child.some_minimum`, which is irrelevant
        #considering that we called `Parent.test`
        if not kwarg['some_arg'] > self.some_minimum:
            raise ValueError('Validation failed')
        return operation(self, *args, **kwargs)
    return _wrapped

So basically, to solve this problem I see two solutions :

preventing the wrappers to be executed when the method was called with super(Child, self)
having a self that is always of the "right" type

Both solutions seem impossible to me ... Do somebody has an idea on how to solve this ? A suggestion ?

Are you looking for some sort of aspect oriented programming support in Python? See http://stackoverflow.com/questions/286958/any-aop-support-library-for-python for some ideas for that. This isn't exactly what you are asking for, rather it's trying to solve your original needs. — Makis, Mar 07 '11 at 12:44
@Space_C0wb0y : because I want to be able to redeclare `test` in a subclass, without having to redecorate it with 5 decorators ! And anyways that wouldn't solve my problem : the way I wrap the methods from the metaclass is exactly equivalent to using decorators. — sebpiq, Mar 07 '11 at 12:51

score 1 · Answer 1 · answered Mar 07 '11 at 12:43

Well, can't you just check if the context is already set in _manage_context? Like this:

def _manage_context(self, *args, **kwargs):
    #Sets the context to 'blabla' before the execution
    if self.context is None:
        self.context = 'blabla'
        returned = operation(self, *args, **kwargs)
        #Cleans the context after execution
        self.context = None
        return returned
    else:
        return operation(self, *args, **kwargs)

Also, this should probably be wrapped in a try-catch block, to ensure resetting of the context in case of exceptions.

That's an idea ... however it is going to break if for some reason the context is set from outside the method. Also, it requires all the other wrappers to check if the context is already set before executing. — sebpiq, Mar 08 '11 at 08:28

score 0 · Answer 2 · answered Jul 18 '15 at 19:41

Ok, first, your "solution" is really ugly, but I suppose you know that. :-) So let's try to answer your questions.

First is an implicit "question": why don't you use Python's context managers? They give you much nicer syntax and error management practically for free. See contextlib module, it can help you greatly. Especially see section about reentrancy.

Then you'll see that people usually have problems when trying to stack context managers. That's not surprising, since to properly support recursion you need a stack of values, not a single value. [You could see the source for some reentrant cm, for example redirect_stdout, to see how it's handled.] So your context_wrapper should either:

(cleaner) keep a list of self.contexts, append to it when entering context, and pop from it when exiting. That way you always get your context.
(more like what you want) keep a single self.context, but also a global value DEPTH, increased by one on entering, decreased by one on exiting, and self.context being reset to None when DEPTH is 0.

As for your second question, I must say I don't quite understand you. self is of the right type. If A is subclass of B, and self is instance of A, then it is also instance of B. If self.some_minimum is "wrong" whether you consider self an instance of A or of B, that means that some_minimum is not really an instance attribute of self, but a class attribute of A or B. Right? They can be freely different on A and on B, because A and B are different objects (of their metaclass).

score 0 · Accepted Answer · answered Mar 07 '11 at 15:15

Actually I have found out a way to prevent the wrappers to be executed when the method was called with super(Child, self) :

class MyType(type):
    def __new__(cls, name, bases, attrs):
        #wraps the 'test' method to automate context management and other stuff
        new_class = super(MyType, cls).__new__(cls, name, bases, attrs)
        new_class.test = cls.other_wrapper(new_class.test, new_class)

    @classmethod
    def other_wrapper(cls, operation, new_class):
        def _wrapped(self, *args, **kwargs):
            #DO something with self and *args and **kwargs ...
            #ONLY if self is of type *new_class* !!!
            if type(self) == new_class:
                pass #do things
            return operation(self, *args, **kwargs)
        return _wrapped

That way, when calling :

super(Child, self).a_wrapped_method

The wrapping code is by-passed !!! That's quite hackish, but it works ...

Python : metaclass + wrapped methods + inheritance = problems

3 Answers3