1

I am new to the more advanced features of Python like decorators.

I am unable to understand how the Python interpreter actually understands where to put the original function object in a decorator.

Lets look at an example: Examples taken from here.

Simple decorator with no arguments:

def call_counter(func):
    def helper(*args, **kwargs):
        helper.calls += 1
        return func(*args, **kwargs)
    helper.calls = 0

    return helper

@call_counter
def succ(x):
    return x + 1

This makes perfect sense if we can assume that the first/only argument to the decorator call_counter(func) is the function object that needs to wrapped ie. in this case succ() function.

But things become inconsistent when you are talking about "decorators with parameters". Look at the example below:

Decorator with one argument:

def greeting(expr): # Shouldn't expr be the function here ? Or at least isn't there suppose to be another parameter.
    def greeting_decorator(func): # How does Python know to pass the function down here ?
        def function_wrapper(x):
            print(expr + ", " + func.__name__ + " returns:")
            func(x)
        return function_wrapper
    return greeting_decorator

@greeting("Hello")
def foo(x):
    print(42)

foo("Hi")

Now we know Python has no concept of data-types, so function parameters give no information about what type of object they will contain.

Am I correct ?

Having said that lets look at the line from the above example:

def greeting(expr):

If for decorators the first argument is the function to be wrapped then by that logic expr should point to foo() right ? Otherwise there should be at least two parameters in greeting(), like:

def greeting(func, expr):

But instead Python can "magically" understand that the inner function needs to be passed the function reference:

def greeting(expr): 
    def greeting_decorator(func): # How is it correctly put one level down ?

The code has no datatypes or type information specified, so how is it that for decorators without arguments the function is passed as the first argument and for decorators with arguments the function is passed to the inner function ?

How can the interpreter detect that ?

What is going on here ?

This seems like "magic" to me.

What happens if I have 5 or 6 levels of nested functions ?

I am pretty sure I am missing something pretty basic here.

Thanks.

ng.newbie
  • 2,807
  • 3
  • 23
  • 57
  • 1
    "Now we know Python has no concept of data-types," Absolutely incorrect, Python is a strongly type language, with well-defined data types. Every object has a type, and knows it's type, and that type is introspectable at runtime. Although, that isn't how the decorator knows. Note, Python is dynamically typed, not statically typed, which may be what you meant. – juanpa.arrivillaga Jul 24 '20 at 23:24
  • 2
    You may want to see my answer to get a further understanding: https://stackoverflow.com/questions/51891951/python-decorator-typeerror-missing-1-required-positional-argument/51892248#51892248. To summarize, `greeting("Hello")` returns a function which accepts `foo` as the parameter, you definitely can nest multiple levels as many as you want. – Sraw Jul 24 '20 at 23:24
  • 2
    The first argument *to `greeting("Hello")`* is the decorated function. The first argument to `greeting` is `"Hello"`. – user2357112 Jul 24 '20 at 23:25
  • @Sraw Ok then how does Python know when to stop calling the function. I mean first it calls `greeting()` then it gets back `greeting_decorator()` then it calls that function. **How does Python know when to stop ?** – ng.newbie Jul 24 '20 at 23:32
  • 1
    @ng.newbie **you** call `greeting("Hello")`, that returns a decorator, in this case, the inner `greeting_decorator ` which when applied with the `@decorator` syntax, gets turned into `function = greeting_decorator(function)`, that's just how `@decorator` is syntax is defined to work – juanpa.arrivillaga Jul 24 '20 at 23:33
  • @user2357112supportsMonica I am sorry but could you be a little more clear ? I did not understand. – ng.newbie Jul 24 '20 at 23:33
  • @juanpa.arrivillaga Ok what if `greeting("Hello")` returned another function which then had another function which returned the decorator ? **I mean how would you handle multiple levels of nesting ?** – ng.newbie Jul 24 '20 at 23:37
  • Putting `@greeting("Hello")` above a function definition for `foo` is simply an alternative syntax for putting `foo = greeting("Hello")(foo)` below it. Do you understand how *that* syntax knows what to pass to what? – user2357112 Jul 24 '20 at 23:38
  • Just as user2357112 said, it is just a normal function call, there is nothing else. Multiple levels, multiple function calls, that's it. – Sraw Jul 24 '20 at 23:39
  • @user2357112supportsMonica Yes, the imperative style is pretty clear, but I am having trouble matching that up with the declarative/syntactic-sugar style of it. – ng.newbie Jul 24 '20 at 23:41
  • 1
    @ng.newbie then *you* would have to do something like `@level1('foo')('bar')`, the `@` syntax simply calls the *result of the expression* with the function. It doesn't know how much nesting is involved... – juanpa.arrivillaga Jul 24 '20 at 23:42
  • @juanpa.arrivillaga Ok so just to test my understanding: `@level1('foo')('bar')` : I am responsible for getting the function that is used as the decorator, I mean I call `level1` with `foo` and then its result with `bar`, which in turn gives the decorator. **It is my job to traverse the call chain, until the decorator function is reached.** Am I correct ? – ng.newbie Jul 24 '20 at 23:46
  • [This answer](https://stackoverflow.com/a/1594484/523612) is basically everything there is to know about decorators. – Karl Knechtel Jul 24 '20 at 23:47
  • @juanpa.arrivillaga You may want to put everything you said into an answer. – ng.newbie Jul 24 '20 at 23:55

1 Answers1

1

Python evaluates the expression after the @ and uses the result as the decorator function.

Python calls the __call__ method of the object that is the decorator with the function as argument.

using

@call_counter
def succ(x):
    return x + 1

callcounter is the object looked for __call__ to give the argument func

If you use

@greeting("Hello")
def foo(x):
    print(42)

greeting("Hello") is evaluated and its result is an object that Python uses the __call__ method with the func argument.

rioV8
  • 24,506
  • 3
  • 32
  • 49
  • 2
    The stuff about `__call__` is technically accurate but not particularly relevant. – user2357112 Jul 24 '20 at 23:29
  • 1
    I agree, it's probably more confusing than enlightening to someone trying to understand parameterized decorators. – juanpa.arrivillaga Jul 24 '20 at 23:30
  • @juanpa.arrivillaga : I made a little adjustment, But not telling what happens behind the scene can cause more confusion. A function is an object that can be called with the `()` operator. – rioV8 Jul 24 '20 at 23:37