How do Python tell “this is called as a function”?

Question

A callable object is supposed to be so by defining __call__. A class is supposed to be an object… or at least with some exceptions. This exception is what I'm failing to formally clarify, thus this question posted here.

Let A be a simple class:

class A(object):

    def call(*args):
        return "In `call`"

    def __call__(*args):
        return "In `__call__`"

The first function is purposely named “call”, to make clear the purpose is the comparison with the other.

Let's instantiate it and forget about the expression it implies:

a = A() # Think of it as `a = magic` and forget about `A()`

Now what's worth:

print(A.call())
print(a.call())
print(A())
print(a())

Result in:

>>> In `call`
>>> In `call`
>>> <__main__.A object at 0xNNNNNNNN>
>>> In `__call__`

The output (third statement not running __call__) does not come as a surprise, but when I think every where it is said “Python class are objects”…

This, more explicit, however run __call__

print(A.__call__())
print(a.__call__())

>>> “In `__call__`”
>>> “In `__call__`”

All of this is just to show how finally A() may looks strange.

There are exception in Python rules, but the documentation about “object.call” does not say a lot about __call__… not more than that:

3.3.5. Emulating callable objects
object.__call__(self[, args...])
Called when the instance is “called” as a function; […]

But how do Python tell “it's called as a function” and honour or not the object.__call__ rule?

This could be a matter of type, but even type has object as its base class.

Where can I learn more (and formally) about it?

By the way, is there any difference here between Python 2 and Python 3?

----- %< ----- edit ----- >% -----

Conclusions and other experiments after one answer and one comment

Update #1

After @Veedrac's answer and @chepner's comment, I came to this other test, which complete the comments from both:

class M(type):

    def __call__(*args):
        return "In `M.__call__`"

class A(object, metaclass=M):

    def call(*args):
        return "In `call`"

    def __call__(*args):
        return "In `A.__call__`"

print(A())

The result is:

>>> In `M.__call__`

So it seems that's the meta‑class which drives the “call” operations. If I understand correctly, the meta‑class does not matter only with class, but also with classes instances.

Update #2

Another relevant test, which shows this is not an attribute of the object which matters, but an attribute of the type of the object:

class A(object):

    def __call__(*args):
        return "In `A.__call__`"

def call2(*args):
    return "In `call2`"


a = A()

print(a())

As expected, it prints:

>>> In `A.__call__`

Now this:

a.__call__ = call2

print(a())

It prints:

>>> In `A.__call__`

The same a before the attribute was assigned. It does not print In call2, it's still In A.__call__. That's important to note and also explain why that's the __call__ of the meta‑class which was invoked (keep in mind the meta‑class is the type of the class object). The __call__ used to call as function, is not from the object, it's from its type.

`A.__call__()` does **not** work the way you demonstrate. It will **not** print `In __call__`. — Martijn Pieters, Sep 10 '14 at 17:25
That's at least what it do with Python 3.4 (are you using an older Python version?) — Hibou57, Sep 10 '14 at 17:26
**Instances** of `A` are callable; special methods are looked up on the type of an object; you cannot make `__call__` an attribute of an instance for example. — Martijn Pieters, Sep 10 '14 at 17:26
Only because you used `*args` for the signature. Print out what `args` is and you'll see the difference there. You are accessing the *function object* there. — Martijn Pieters, Sep 10 '14 at 17:27
And you know what? Functions are callables. It doesn't matter what you named them when you call them directly. — Martijn Pieters, Sep 10 '14 at 17:29

Veedrac · Accepted Answer · 2014-09-10T17:54:52.197

5

x(*args, **kwargs) is the same as type(x).__call__(x, *args, **kwargs).

So you have

>>> type(A).__call__(A)
<__main__.A object at 0x7f4d88245b50>

and it all makes sense.

chepner points out in the comments that type(A) == type. This is kind-of wierd, because type(A)(A) just gives type again! But remember that we're instead using type(A).__call__(A) which is not the same.

So this resolves to type.__call__(A). This is the constructor function for classes, which builds the data-structures and does all the construction magic.

The same is true of most dunder (double underscore) methods, such as __eq__. This is partially an optimisation in those cases.

edited Sep 10 '14 at 17:54

answered Sep 10 '14 at 17:27

Veedrac

58,273
15
112
169

Probably worth pointing out explicitly that `type(A)` is `type`. – chepner Sep 10 '14 at 17:47
So that's because it has `type` as an ancestor? That would mean that's not just defining `__call__` which makes something callable and there is more involved. – Hibou57 Sep 10 '14 at 17:47
1

No, since `type` is itself a type, `type.__call__` is called when ever you attempt to call an *instance* of `type` as a function. Since the class `A` is an instance of `type`, `type.__call__` is what gets called when you write `A()` (specifically, it calls `A.__new__`). There's no special exception that makes classes callable; it's just a consequence of classes being instances of their metaclass. – chepner Sep 10 '14 at 17:52
@chepner, so that's a special rule the documentation forget to mention? – Hibou57 Sep 10 '14 at 17:53
@Hibou57 Not really. Does my alternative try at explaining this (I just edited my post) help? – Veedrac Sep 10 '14 at 17:55
1

Another thing to keep in mind is that the metaclass of `A` (`type`) is distinct from the _parent_ of `A` (`object`). (It gets a _little_ confusing since `object` is the parent of `type` and `type` is the metaclass of `object`, and `type` has no metaclass and `object` has no parent. Perhaps it's best to ignore this parenthetical.) – chepner Sep 10 '14 at 17:58
A *little* confusing? TBH you don't really need to learn about metaclasses. I've never needed them, and the explanation for how `__call__` works can mostly avoid mentioning them. – Veedrac Sep 10 '14 at 18:00
@Veedrac, if it really implies meta‑classes, I don't mind learning more on this. I precisely came to this question during personal investigation about the idea of using Python for meta‑programming and program analysis (with possible program generation to another target language from Python program seen as “expression”). – Hibou57 Sep 10 '14 at 18:05
Thanks, I got it: if I define `__call__` on a meta‑class, this `__call__` gets invoked. That's indeed the meta‑class which drive the operation of the call. @chepner was right too (you are both right). I will edit my initial question to add the additional test I made. – Hibou57 Sep 10 '14 at 18:10
You don't need to know about metaclasses to explain `__call__` in general, but it helps a great deal for explaining why `A()` isn't some special case for classes that creates instances. (BTW, I'm not positive that `type` is the metaclass of `object`; as a built-in type, i don't think metaclasses apply.) – chepner Sep 10 '14 at 18:10
2

Annnnd.... one last comment. If `f` is a function, `f()` is not special, either. It's simply short for `type(f).__call__(f)`. There's no builtin name that references the type, but there is one in the `types` module: `import types; types.FunctionType.__call__(f)`. – chepner Sep 10 '14 at 18:17
@Hibou57 Yes. I think you've got it. – Veedrac Sep 10 '14 at 18:30
Yes, I think too, and thanks to you :-). I added a second edit to my original question, a short test which rewords your explanations. – Hibou57 Sep 10 '14 at 18:36

How do Python tell “this is called as a function”?

Conclusions and other experiments after one answer and one comment

Update #1

Update #2

1 Answers1

Linked