2

I set up a training set with X of shape (batch_size, 50) and Y of shape (batch_size, 10(sequence length), 10(output vector)).

The Keras Documentation of the LSTM cell says, that a 3D input is required, but Sequence 2 Sequence models work and do exactly this. Is there another common way to address this task?

model = Sequential()
model.add(LSTM(50, input_shape=(50,)))
model.add(TimeDistributed(Dense(10, activation='softmax')))

I get the error: "ValueError: Input 0 is incompatible with layer lstm_1: expected ndim=3, found ndim=2"

Moritz Blum
  • 93
  • 7

2 Answers2

1

This return only the last step of the Sequence, you should set the argument: return_sequences=True

For the following model: 

 model = Sequential()
 model.add(LSTM(50, input_shape=(1, 50), return_sequences=True))
 model.add(TimeDistributed(Dense(10, activation='softmax')))

I Have:

Layer (type)                 Output Shape              Param #   
=================================================================
lstm_3 (LSTM)                (None, 1, 50)             20200     
_________________________________________________________________
time_distributed_2 (TimeDist (None, 1, 10)             510       
=================================================================
Total params: 20,710
Trainable params: 20,710
Non-trainable params: 0
_________________________________________________________________
Benjamin Breton
  • 1,388
  • 1
  • 13
  • 42
  • You are right, but this has no effect on the input. – Moritz Blum Apr 21 '19 at 13:04
  • Have you tried performing a reshape on your input and setting **input_shape=(50, 1)** ? – Benjamin Breton Apr 21 '19 at 13:08
  • I have reshaped each batch to (batch_size, 1, 50) and set input_shape=(1, 50). Now I get an error from the TimeDistributed layer: "ValueError: Error when checking target: expected time_distributed_1 to have shape (1, 10) but got array with shape (10, 10)" – Moritz Blum Apr 21 '19 at 13:35
0

Changing the input_shape to (1, 50) and adding a

model.add(RepeatVector(10, input_shape=(50, )))

as first layer, fixes the problem. The additional RepeatVector makes the input visible to the network at all times step.

A good overview over Keras LSTM nets is: How to use return_sequences option and TimeDistributed layer in Keras?

Moritz Blum
  • 93
  • 7