1

I am learning to use Keras LSTM model. I have looked at this tutorial, this tutorial and this tutorial and am feeling unsure about my understanding of LSTM model's input shape. My question is if one is to shape one's data like the first tutorial (8760, 1, 8) and the data is inputted to the network 1 timestep at a time i.e. the input_shape=(1, 8) does the network learn the temporal dependencies between samples?

wwwwwwwwww
  • 123
  • 11
  • When those links go dead, no one will have any idea what your question is about. – Rob May 19 '19 at 01:22

1 Answers1

1

It only makes sense to have batches of 1 timestep when you're using stateful=True. Otherwise there is no temporal dependency, as you presumed.

The difference is:

  • stateful=False, input_shape=(1,any):
    • first batch of shape (N, 1, any): contains N different sequences of length 1
    • second batch: contains another N different sequences of length 1
    • total of the two batches: 2N sequences of length 1
    • more batches: more independent sequences
    • yes, there is no point in using steps=1 when stateful=False
  • stateful=True, input_shape=(1,any):
    • first batch of shape (N, 1, any): contains the first step of N different sequences
    • second batch: contains the second step of the same N sequences
    • total of the two batches: N sequences of length 2
    • more batches = more steps of the same sequences, until you call model.reset_states()

Usually, it's more complicated to handle stateful=True layers, and if you can put entire sequences in a batch, like input_shape=(allSteps, any), there is no reason to turn stateful on.

If you want a detailed explanation of RNNs on Keras, see this answer

Daniel Möller
  • 84,878
  • 18
  • 192
  • 214
  • If I have a data set comprising 1000 individuals, each has a sequence of 20 steps and 10 features, currently, the data has the shape of 20,000 * 10 (individuals are appended one after another row-wise). How should I shape this data for input into LSTM? – wwwwwwwwww Nov 29 '18 at 23:23
  • `(1000 individuals, 20 steps, 10 features)` - I don't know what you mean by individuals are appended one after another row-wise. – Daniel Möller Nov 29 '18 at 23:33
  • oh, I was trying to show how I got the length 20,000. Sorry for the confusion. Thanks heaps for your help. – wwwwwwwwww Nov 29 '18 at 23:37
  • :) -- If you consider this answered your question, please mark it as answered – Daniel Möller Nov 29 '18 at 23:49
  • @ Daniel from reading the link you provided, I have tried to implement a many-to-many model for my training data and a one-to-many model for prediction using my test data. The model runs ok, but the prediction is not very good. Can I check with you regarding how I am setting up the model? Thank you – wwwwwwwwww Dec 04 '18 at 01:22