2

This was my python code:

 from tensorflow.keras.models import Sequential
 from tensorflow.keras.layers import SimpleRNN

 Model = Sequential([
      SimpleRNN(2, input_shape=(2,2))
 ])

 print(Model.summary())

The output was:

Model: "sequential_23"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
simple_rnn_10 (SimpleRNN)    (None, 2)                 10        
=================================================================
Total params: 10
Trainable params: 10
Non-trainable params: 0
_________________________________________________________________
None

I don't understand why the Number of parameters is 10.

desertnaut
  • 57,590
  • 26
  • 140
  • 166
  • This question is not about programming, but about how RNNs are built and parameters are defined. I think you'll get better answers at stats.SE or datascience.SE – Davidmh Sep 18 '20 at 14:36
  • Does this answer your question? [Number of parameters for Keras SimpleRNN](https://stackoverflow.com/questions/50134334/number-of-parameters-for-keras-simplernn) – MBT Sep 20 '20 at 11:22

1 Answers1

1

A simple recurrent layer is defined by:

enter image description here

In your code, you have specified input_shape=(2,2) which indicates |x|=2. You also set |h|=2 and have no output layer definied. This leads to the following model:

enter image description here

Since everything is fully connected in the feed-forward input (black connections) and recurrent input (blue connections), you get 2x2 matrices (weight shape in orange). The bias is a single value that is fully connected to each h unit, i.e. there is one weight per unit (in total 2, see green connections).

So in summary:

W_xh is the input matrix: 2x2=4 parameters

W_hh is the recurrent matrix: 2x2=4 parameters

b_h is the bias: 2 parameters

So in total, you have 10 parameters.

runDOSrun
  • 10,359
  • 7
  • 47
  • 57
  • 1
    Can you please show with a diagram. It would be much more helpful and understandable with a diagram. –  Sep 20 '20 at 06:05