How to make simple data recall in keras neural networks

Question

I'm trying the get a hang of keras and I'm trying to get basic time series prediction working. My input is a list of random ints between 0 and 10 such as:[1,3,2,4,7,5,9,0] and my labels are the same as the input but delayed such as: [X,X,1,3,2,4,7,5] and I'm trying to have my model learn this relationship of remembering past data points. My code is:

labels = keras.utils.to_categorical(output, num_keys)

model = keras.Sequential([
    keras.layers.LSTM(10),
    keras.layers.Dense(10, activation='relu'),
    keras.layers.Dense(10, activation=tf.nn.softmax)
    ])

model.compile(optimizer=tf.train.AdamOptimizer(),
              loss=tf.keras.losses.categorical_crossentropy,
              metrics=['accuracy'])

model.fit(input, labels, epochs=30, verbose=2,shuffle=False)

and I get the error:ValueError: Please provide as model inputs either a single array or a list of arrays. You passed: x=[7, 6,...

I've tried reformating my input with:

input=numpy.array([[i,input[i]]for i in range(len(input))])
input=numpy.reshape(input,input.shape+(1,))

and adding input_shape=input.shape[1:] to my LSTM layer and that throws no errors but the accuracy is no better then just blind guessing

This seems like that kind of thing that could be trivial but I'm clearly missing something.

from keras import michael · Accepted Answer · 2018-08-24T05:36:53.950

1

With keras.layers.LSTM(10), you need to include the input data shape: keras.layers.LSTM(10, input_shape = (input.shape[1], input.shape[2])).

Keras is expecting the input data shaped as [instances, time, predictors] and since you don't have any additional predictors, you may need to reshape your input data to input.reshape(input.shape[0], input.shape[1], 1).

Keras will infer the data shapes for the next layers, but the first layer needs the input shape defined.

edited Aug 24 '18 at 05:36

answered Aug 24 '18 at 05:30

from keras import michael

721
1
10
23

This is helpful but now I'm getting an error because input.shape only has a length of 2 for [2] is out of range and when I give LSTM input_shape = input.shape that works but then throws the error: expected lstm_input to have 3 dimensions but got array with shape(1000,1) – Supamee Aug 27 '18 at 15:29
That error often means your input data is not shaped to three dimensions (often, you need to explicitly make the third dimension 1). Have a look at my code at: https://stackoverflow.com/questions/47251061/keras-lstm-necessary-shape-when-using-return-sequences-true/52032046#52032046. – from keras import michael Aug 27 '18 at 17:11
Also, I am not sure if this is related to the error you are getting but you seem to want to model a numeric series "My input is a list of random ints between 0 and 10", but your Keras code is structured for modeling a categorical outcome. – from keras import michael Aug 27 '18 at 17:23
I was modeling each number as a category to avoid things like 5 being between 4 and 6. My labels are sparse – Supamee Aug 27 '18 at 19:20
Fair enough. Were you able to adapt the code from my other post for your question? – from keras import michael Aug 27 '18 at 19:31
I was able to adapt your code from the other question, and it runs but does not learn. – Supamee Aug 27 '18 at 20:58
1

Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/178889/discussion-between-supamee-and-from-keras-import-michael). – Supamee Aug 27 '18 at 21:04

How to make simple data recall in keras neural networks

1 Answers1