5

i am trying to build a deep learning network based on LSTM RNN here is what is tried

from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
from keras.layers import Embedding
from keras.layers import LSTM
import numpy as np

train = np.loadtxt("TrainDatasetFinal.txt", delimiter=",")
test = np.loadtxt("testDatasetFinal.txt", delimiter=",")

y_train = train[:,7]
y_test = test[:,7]

train_spec = train[:,6]
test_spec = test[:,6]


model = Sequential()
model.add(LSTM(32, input_shape=(1415684, 8),return_sequences=True))
model.add(LSTM(64, input_dim=8, input_length=1415684, return_sequences=True))
##model.add(Embedding(1, 256, input_length=5000))
##model.add(LSTM(64,input_dim=1, input_length=10, activation='sigmoid',
##               return_sequences=True, inner_activation='hard_sigmoid'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='rmsprop')

model.fit(train_spec, y_train, batch_size=2000, nb_epoch=11)
score = model.evaluate(test_spec, y_test, batch_size=2000)

but it gets me the following error

ValueError: Error when checking input: expected lstm_1_input to have 3 dimensions, but got array with shape (1415684, 1)

Here is a sample from the dataset

(Patient Number, time in millisecond, accelerometer x-axis,y-axis, z-axis,magnitude, spectrogram,label (0 or 1))

1,15,70,39,-970,947321,596768455815000,0
1,31,70,39,-970,947321,612882670787000,0
1,46,60,49,-960,927601,602179976392000,0
1,62,60,49,-960,927601,808020878060000,0
1,78,50,39,-960,925621,726154800929000,0

in the dataset i am using the only the spectrogram as input feature and the label (0 or 1) as the output the total traing samples is 1,415,684

Maxim
  • 52,561
  • 27
  • 155
  • 209
Hadeer El-Zayat
  • 281
  • 5
  • 20

1 Answers1

8

Your main mistake is misunderstanding how LSTM (or any RNN, actually) works and what it accepts as an input. A single training example for LSTM network consists of a sequence and a label. For example, this...

1,15,70,39,-970,947321,596768455815000,0
1,31,70,39,-970,947321,612882670787000,0
1,46,60,49,-960,927601,602179976392000,0
1,62,60,49,-960,927601,808020878060000,0
1,78,50,39,-960,925621,726154800929000,0

... is a sequence of length 5, with 8 features. The label for the whole sequence is the label column at the next row. Note that this is just one example; a batch means several such sequences and labels.

Now, concerning Keras, from this answer:

LSTM layer is a recurrent layer, hence it expects a 3-dimensional input (batch_size, timesteps, input_dim).

Let's look closely at your specification: input_shape=(1415684, 8) tells keras to expect sequences of length 1415684, in which each item has 8 features. And all this not accounting for the batch size, which is 2000.

This clearly won't work, because 1415684 is way too long LSTM sequence. Empirical evidence shows that LSTM can learn up to 100 time steps, so feeding larger sequences won't make it learn better. Not to mention that it's not memory and time efficient.

What you should do instead is choose a smaller timesteps parameter, say timesteps=64, and split your data into chunks of timesteps subsequent rows. Chunks may intersect. The batch of those means batch_size * timesteps rows in total, each with 8 columns. The y_train should contain the ground truth for each training sequence. Keras won't do this preparation step, so you'll have to do it manually.

Maxim
  • 52,561
  • 27
  • 155
  • 209