4

I'm trying to train LSTM model in Keras using data of variable timestep, for example, the data looks like:

x1 : [[0,0], [0,0], [0,0]]  
x2 : [[0,0], [0,0]]
x3 : [[0,0], [0,0], [0,0], [0,0]]

x1's timestep is 3, x2's timestep is 2 and x3's timestep is 4

Here is the definition of lstm layer:

model.add(LSTM(256, input_shape=(None, 2), return_sequences=True))

I can train this model using 

model.fit(x1, y1)
model.fit(x2, y2)
model.fit(x3, y3)

It works fine. but in this way, only 1 data can be used to train the model.

Because x1, x2 and x3 has different shape, I can't stack them to a numpy array like: 

np.hstack((x1, x2, x3))

So do I have to train the model using data one by one? Is there some way to train this model in batch? Thank you all for helping me with it :)

sunnwmy
  • 143
  • 2
  • 9
  • 1
    Possible duplicate of [How do I create a variable-length input LSTM in Keras?](https://stackoverflow.com/questions/38189070/how-do-i-create-a-variable-length-input-lstm-in-keras) – erip Mar 04 '18 at 15:44
  • @erip Hello, my question is how can I train this kind of model in batch? – sunnwmy Mar 04 '18 at 15:48
  • In order to *create* a batch, the sequences must be padded to the common length – Maxim Mar 04 '18 at 17:04
  • @Maxim So I would have no choice but to train the model using data one by one if I want the input of the model to have variable timesteps? – sunnwmy Mar 05 '18 at 00:56
  • You can pad the data to create a batch, then ignore the padding with a mask during training. The padding will have no effect if you do it right. – iga Mar 16 '18 at 03:06
  • @iga okay, it seems this is the only option, thank you – sunnwmy Mar 18 '18 at 16:38

1 Answers1

0

This is maybe a bit late, but for those who stumble on this now, Keras provides something called Masking: Hereby, you take a masking value, e.g. 0 and all timesteps with only 0 as value will be skipped.

https://www.tensorflow.org/guide/keras/masking_and_padding

Steffen4408
  • 3
  • 2