2

I'm trying to understand the relationship between a simple Perceptron and a neural network one gets when using the keras Sequence class.

I learned that the neural network perceptron looks as such:

enter image description here

Each "node" in the first layer is one of the features of a sample x_1, x_2,...,x_n

Could somebody explain the jump to the neural network I find in the Keras package below? Since the input layer has four nodes, does that mean that network consists of four of the perceptron networks? enter image description here

Helen
  • 533
  • 12
  • 37

1 Answers1

2

There is seem to be misunderstanding on what a perceptron is. A perceptron is a single unit that multiplies the inputs with weights, sums them up and applies an activation function: Perceptron

Now the diagrams you have are called multi-layer perceptrons (MLP) and consist of a stack of perceptrons organised in layers, wiki. In Keras, there is no explicit notion of a perceptron but of a layer of perceptrons implemented as a Dense layer because the layers are densely connected, ie every output is connected to every input between layers. The second diagram would correspond to:

model = Sequential()
model.add(Dense(4, activation='sigmoid', input_dim=3))
model.add(Dense(4, activation='sigmoid'))
model.add(Dense(1, activation='sigmoid'))

assuming you have sigmoid activation. In this case, the input layer is implicit by specifying the input_dim=3 and the final layer would be the output layer.

nuric
  • 11,027
  • 3
  • 27
  • 42
  • are you sure the second diagram would correspond to that? First, *input_dim* could be anything, right? and also second line should be "...Dense(3..." ? – Helen May 17 '18 at 15:59
  • 1
    Yes, I'm sure, you are confusing input layers and hidden layers. The input dim for the second diagram is 3 because you have 3 input nodes. Then these connect to the first hidden layer which has 4 perceptrons hence Dense(4..) – nuric May 17 '18 at 16:23
  • Aha, I see! So the second diagram is a neural network (MLP?) that takes three-dimensional data, correct? – Helen May 17 '18 at 16:36
  • Yes indeed both diagrams are neural networks (MLPs) and every column / layer is like a vector of that many dimensions. In Keras you specify that output dimension for a given layer. – nuric May 17 '18 at 16:48
  • Thank you! I think I'm getting it. I'm just also confused that the answer here under "*dim*"(https://stackoverflow.com/questions/44747343/keras-input-explanation-input-shape-units-batch-size-dim-etc) states that even though one puts input_dim = 3, the data is "one-dimensional" – Helen May 17 '18 at 17:07
  • That is because when you have 1 dimension, you can specify your `input_dim` as a scalar. In Keras, `input_shape=(3,)` is equal to `input_dim=3`. – nuric May 17 '18 at 17:44
  • I’m sorry, I dont get it, a scalar is always dim = 1 in my head. Thanks fir trying tho :) – Helen May 17 '18 at 17:55
  • That's it then, no a scalar is dimension 0, have a look at how [tensorflow](https://www.tensorflow.org/versions/r1.2/programmers_guide/dims_types) describes things. – nuric May 17 '18 at 17:57