1

I am trying to create a hybrid model which is consists of EfficientNetB7 and LSTM.

# pretrained model act as a feature extractor
Effnet=tensorflow.keras.applications.EfficientNetB7( input_shape=(IMG_SIZE,IMG_SIZE,3), include_top=False,weights="imagenet",pooling="avg")
Effnet.trainable = False
x = Flatten()(Effnet.output)
x=(BatchNormalization())(x)
#add two LSTM Layers
x=LSTM(8,input_shape=(IMG_SIZE,IMG_SIZE,3),return_sequences=False)(x)
x=LSTM(8)(x)
x=(BatchNormalization())(x)
#add two fully connected dense layers 1024 as my model 
x=Dense(1024)(x)
x=(BatchNormalization())(x)
x=Activation('relu')(x)
x=Dense(1024)(x)
x=(BatchNormalization())(x)
x=Activation('relu')(x)
x = Dense(NUM_CLASSE)(x)
x=(BatchNormalization())(x)
prediction =Activation('softmax')(x)
model = Model(inputs=Effnet.input, outputs=prediction)
model.summary()

But it gives me the following error

and the EfficientNetB7 is average pooling is, I think it is causing the problem, how do I remove it?

Average pooling layer

ValueError: Input 0 of layer "lstm_6" is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 2560)

How can i fix it, please? Thank you, Regards!

Beba.S
  • 31
  • 1
  • 11

1 Answers1

0

The problem is on this line:

x=LSTM(8,input_shape=(IMG_SIZE,IMG_SIZE,3),return_sequences=False)(x)

You defined that the LSTM layers expect input of dimension 3. However, that only hold for the very beginning of your network, which flows into EfficientNetB7. When you have the last output from EfficientNet, you flatten it and get a 1D tensor.

The error message is actually pretty straightforward.

expected ndim=3, found ndim=2. Full shape received: (None, 2560)

2560 comes from flattening the features, and the first dimension is the one for batch size.

You must correct the input to your LSTM layer. If you do not specify anything, keras might just figure it out itself.

Berkay Berabi
  • 1,933
  • 1
  • 10
  • 26
  • the average Pooling layer which is the last layer in efficientnet model converts the input shape to 1D can i remove the pooling layer , to let the LSTM take 3D or not and how or there is another solution ? – Beba.S Aug 14 '22 at 21:37
  • no you cant do that. See this post: https://stackoverflow.com/questions/48475255/python-lstm-based-rnn-required-3d-input – Berkay Berabi Aug 15 '22 at 07:42