writing custom python decorators, and the magic of how they init

Question

I am reading a very clean piece of code at http://mrcoles.com/blog/3-decorator-examples-and-awesome-python/, but the way it initializes confuses me. I see this class decorator taking 'object', but when it runs init, it throws view_func into itself. With view_func not declared anywhere except in init, if it subclasses object, how did it know that view_func was the entire function it's decorating, and that request is the HTTP request?

from functools import wraps

class my_decorator(object):

   def __init__(self, view_func):
        self.view_func = view_func
        wraps(view_func)(self)

    def __call__(self, request, *args, **kwargs):
        # maybe do something before the view_func call
        response = self.view_func(request, *args, **kwargs)
        # maybe do something after the view_func call
        return response

# how to use it...
def foo(request): return HttpResponse('...')
foo = my_decorator(foo)

# or...
@my_decorator
def foo(request): return HttpResponse('...')

It definitely works, I'm just lost on how it's working exactly. In my logger.py:

class log_decorator(object):
    logpath = "/home/me/logs"

    def __init__(self, func):
        self.func = func
        wraps(func)(self)

    def __call__(self, *args, **kwargs):
        this_path = "{}/{}".format(logpath, self.func.__name__)
        ret = self.func(*args, **kwargs)
        open(this_path, 'w').close()

        if ret:
            with open(this_path, 'a') as myfile:
                myfile.write("Arguments were: {}, {}\n".format(args, kwargs))
                for line in ret:
                    l = str(line)
                    myfile.write(l)
                    myfile.write('\n')
            myfile.close()
        return ret

Mr. Cole's class based style helps me write the recent output of any function to a file in loggers named after the function with just

@log_decorator
def smash_lines(lines):

My exact question would then be how does this class know what view_func and request is, if it is extending object and doesn't require these params? How do class based decorators initialize themselves? Thank you

The class doesn't *know* anything. The original function is stored on `self` as an attribute, so when you call the decorator instance `__call__()` is called, and `self.view_func` exists and can be called in turn. `request` is just an argument to `__call__`; Django passes that in every time. — Martijn Pieters, Jun 15 '15 at 13:30
The original blog post gives you a huge hint; `foo = my_decorator(foo)` is exactly what the decorator syntax (`@my_decorator`) does. The original function is replaced by whatever the `my_decorator()` call returned. Calling a class produces an instance of that class. Calling the instance then as if it was the original function, invokes `__call__`. — Martijn Pieters, Jun 15 '15 at 13:34

dhke · Accepted Answer · 2021-07-28T15:51:36.310

4

I'm not quite sure what gets you confused here, but since there is some Python magic involved, a step-by-step explantion seems to be in order:

my_decorator is a class. my_decorator(foo) is thus not a simple method call but object creation (which, if you want to be totally correct is also a method call, namely to the __call__() method of the class my_decorator). Thus type(my_decorator(foo)) == my_decorator.
my_decorator(foo) calls my_decorator.__init__() which stuffs away the function foo into self.view_func inside this new object.

For example

decorated = my_decorator(foo)
print(foo.view_func)

you will get foo back.

Most important thing to note is probably that the decorator returns an object.

This means that

@my_decorator
def foo(...):
   pass

replaces the original foo() by the return value from my_decorator(foo) which is the my_decorator object we just created. Hence after this line, foo is an object of type my_decorator with the original foo() stuffed inside that object as foo.view_func().

Since my_decorator also emulates function calls by overriding the __call__() method, any call-like operation like foo(request, ...) gets to call my_decorator.__call__(), instead. So __call__() essentially replaced your foo() view function.

edited Jul 28 '21 at 15:51

answered Jun 15 '15 at 13:39

dhke

15,008
2
39
56

1

so foo() really became my_decorator(foo), foo was thrown in the new class as self.view_func, and whatever foo did became whatever my_decorator.__call__ does it seems – codyc4321 Jun 15 '15 at 13:54
1

"my_decorator(foo) is thus not a method call but object creation" : well technically, in Python object instanciation IS a method call - a call to the metaclass `__call__()` method. – bruno desthuilliers Jun 15 '15 at 14:02
@brunodesthuilliers AFAIR it's `__call__()` on the type object itself, not the metaclass. Metaclass resolution is actually done during `type.__new__()`: https://github.com/python/cpython/blob/master/Objects/typeobject.c#L2276 – dhke Jun 15 '15 at 14:14
@dhke try by yourself : write a metaclass, override it's `__call__` method, write class using this metaclass, call the class and you will find out. It's actually `type.__call__(meta, ...)` that calls on `type.__new__(...)` : https://github.com/python/cpython/blob/master/Objects/typeobject.c#L908 – bruno desthuilliers Jun 15 '15 at 14:34
@brunodesthuilliers I'm not quite sure that is different from what I read there. I assume metaclass resolution is done by `_PyType_CalculateMetaclass()`, which is only ever called in `type_new()` which in turn is called from `type_call()`. – dhke Jun 15 '15 at 15:19
@dhke just try what I suggested and find out by yourself. The metaclass `__call__()` **is** called **first** when you instanciate a class. And there's no "metaclass resolution" involved at this stage, it's just plain ordinary attribute lookup rules. Make your metaclass `__call__` not delegating to it's base class and returning `None` and find out if you can still instanciate your class ;) – bruno desthuilliers Jun 15 '15 at 16:09
@brunodesthuilliers: I think you are misunderstanding. My question is *not* *if* metaclass.__call__ is called and its return value used, but rather *who* calls it. And the only code I can find that does the relevant attribute lookup is in referenced in `type_new()`. Trying out what you suggested does not answer that question. – dhke Jun 15 '15 at 16:43
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/80592/discussion-between-bruno-desthuilliers-and-dhke). – bruno desthuilliers Jun 15 '15 at 17:33
Ok if anyone is interested the definitve answer is here : http://stackoverflow.com/questions/30852985/who-calls-the-metaclass – bruno desthuilliers Jun 15 '15 at 20:24

writing custom python decorators, and the magic of how they init

1 Answers1