18

I can't seem to find much documentation on how to interpret the output of get_weights() when running a neural network in Keras. From what I understand, the output is determined by the structure of the network. Therefore, I paste a simplified version of the structure of my network below:

model.add(Dense(5, input_dim=2, activation = linear, use_bias=True, kernel_initializer=Orthogonal))
model.add(Dense(1, use_bias=True))
model.compile(loss='mae', optimizer='adam')

The output of get_weights() after training is:

     [array([[ 0.79376745,  0.79879117,  1.22406125,  1.07782006,  1.24107373],
           [ 0.88034034,  0.88281095,  1.13124955,  0.98677355,  1.14481246]], dtype=float32), 
      array([-0.09109745, -0.09036621,  0.0977743 , -0.07977977,  0.10829113], dtype=float32), 
      array([[-0.72631335],
           [-0.38004425],
           [ 0.62861812],
           [ 0.10909595],
           [ 0.30652359]], dtype=float32), 
      array([ 0.09278722], dtype=float32)]

There are a total of four arrays. What does each represent? Thanks!

Machavity
  • 30,841
  • 27
  • 92
  • 100
Agrippa
  • 475
  • 2
  • 5
  • 13

1 Answers1

24
  • Weights for the first layer (2 inputs x 5 units)
  • Biases for the first layer (5 units)
  • Weights for the second layer (5 inputs x 1 unit)
  • Biases for the second layer (1 unit)

You can always get by layer too:

for lay in model.layers:
    print(lay.name)
    print(lay.get_weights())
Daniel Möller
  • 84,878
  • 18
  • 192
  • 214
  • 6
    This actually makes me realize I misunderstood the shape of the weights in neural networks. I thought that there was one weight per neuron, but in fact the number of weights per neuron depends on the number of connections it has to the previous layer. On the other hand, there is only one bias per neuron. I need to look into the math of neural nets again. Thanks! – Agrippa Oct 18 '17 at 20:05
  • 1
    This helped me a lot at the beginning: http://neuralnetworksanddeeplearning.com/chap1.html – Daniel Möller Oct 18 '17 at 20:07
  • Thanks. Just to confirm, for the first array (i.e the 2x5), the first value (0.7937) is the weight of the first input node connecting to the first neuron, while the last value (1.144) is the weight of the second input node connecting to the 5th neuron. – Agrippa Oct 18 '17 at 20:20
  • Yep :) That's it. – Daniel Möller Oct 19 '17 at 01:20
  • So each layer stores the weights of the "transition" to it, am I right? From the input values to the first hidden layer "transition", the weights are stored in the first hidden layer. And so on until the output layer that stores the weights of the "transition" from the last hidden layer to the output layer. Am I thinking correctly? – Joel Carneiro Jan 09 '19 at 16:59
  • Yes. The layer itself is a transition. When you use functional models you see that more clearly. You have tensors (data) and layers (data transformations) – Daniel Möller Jan 10 '19 at 14:57