2

I saw a Python function calling way in Keras' tutorial like this:

from keras.layers import Input, Dense
from keras.models import Model

# this returns a tensor
inputs = Input(shape=(784,))

# a layer instance is callable on a tensor, and returns a tensor
x = Dense(64, activation='relu')(inputs)

However, I have no idea what is the meaning of the function calling form:

x = Dense(64, activation='relu')(inputs)

Why is there a "(inputs)" outside the bracket of the function "Dense"'s parameters list?

Zhi Lu
  • 527
  • 5
  • 22

1 Answers1

8

Because Dense(...) returns a callable (basically, a function), so it can be called in turn. Here's a simple example:

def make_adder(a):
    def the_adder(b):
        return a + b
    return the_adder

add_three = make_adder(3)
add_three(5)
# => 8

make_adder(3)(5)
# => 8

Here, make_adder(3) returns the function that is defined as

def the_adder(b)
    return 3 + b

then calling that function with argument 5 returns 8. If you skip the step of assigning the return value of make_adder(3) to a separate variable, you get the form that you were asking about: make_adder(3)(5) is the same thing as Dense(64, activation='relu')(inputs) from your question.

EDIT: Technically, Dense is not classified as a function in Python, but as a class; Dense(...) is thus an invocation of a constructor. The class in question defines the __call__ method, which makes objects of this class "callable". Both functions and callable objects can be called by invoking them with an argument list, and the difference between the two does not impact the explanation at all. However, here's an example of a simple callable, that more closely parallels Dense:

class Adder:
    def __init__(self, a):
        self.a = a
    def __call__(self, b):
        return self.a + b

adder_of_three = Adder(3)
adder_of_three(5)
# => 8

Adder(3)(5)
# => 8
Amadan
  • 191,408
  • 23
  • 240
  • 301
  • This example may be misleading, because in this case [Dense](https://github.com/fchollet/keras/blob/master/keras/layers/core.py#L607) is a class that defines `__call__`, but the example shows a factory function. – SethMMorton Nov 08 '16 at 05:26
  • 1
    @SethMMorton: Conceptually, `Dense` is a function that returns a callable (as a constructor of a class defining a `__call__` method). `make_adder` is also a function that returns a callable (by returning a function, which is by definition a callable). They are not that different. Technically, sure - `Dense` is a class, and `make_adder` a function - lots of differences under the hood. Hmm, I might add that though... – Amadan Nov 08 '16 at 05:27
  • Yes, I agree that this is how a class is built under the hood. But for a new user the description may be confusing because if they looked at the source code to `Dense` they would see it defined as a class and not a function. All I am trying to say is that this answer does not connect the dots between a factory function and a callable class, it might make it improved if the connection were made. Although the OP already accepted so maybe it's not worth it. – SethMMorton Nov 08 '16 at 05:35
  • Thank you guys for your answering and discussion! I am clear about it now. – Zhi Lu Nov 08 '16 at 05:36