I have this error "input 0 is incompatible with layer lstm expected ndim=3 found ndim=5"

Question

I am very new in this field. I searched on the internet but I could not find a solution. I am waiting for the help of people who are interested in this field.

My model

def load_VGG16_model():
    base_model = VGG16(weights='imagenet', include_top=False, input_shape=(256,256,3))
    print("Model loaded..!")
    return base_model

Summary of the model

load_VGG16_model().summary()

Adding Layers

def action_model(shape=(30, 256, 256, 3), nbout=len(classes)):
    convnet = load_VGG16_model()

    model = Sequential()
    model.add(TimeDistributed(convnet, input_shape=shape))
    model.add(LSTM(30,return_sequences=True,input_shape=(30,512))) # the error shows this line.
    top_model.add(Dense(4096, activation='relu', W_regularizer=l2(0.1)))
    top_model.add(Dropout(0.5))
    top_model.add(Dense(4096, activation='relu', W_regularizer=l2(0.1)))
    top_model.add(Dropout(0.5))
    model.add(Dense(nbout, activation='softmax'))
    return model

model.add(LSTM(30,return_sequences=True,input_shape=(30,512))) ==> the error shows this line.

Answer 1

Your problem is similar to this one Building CNN + LSTM in Keras for a regression problem. What are proper shapes?

Using reshape layer before the LSTM should work fine for you

def action_model(shape=(256, 256, 3), nbout=len(classes)):
    convnet = load_VGG16_model()

    model = Sequential()
    model.add(convnet)
    model.add(tf.keras.layers.Reshape((8*8, 512))) # Shape comes from the last output of covnet
    model.add(LSTM(30,return_sequences=True,input_shape=(8*8,512))) # the error shows this line.
    top_model.add(Dense(4096, activation='relu', W_regularizer=l2(0.1)))
    top_model.add(Dropout(0.5))
    top_model.add(Dense(4096, activation='relu', W_regularizer=l2(0.1)))
    top_model.add(Dropout(0.5))
    model.add(Dense(nbout, activation='softmax'))
    return model