5

I have seen examples of building an encoder-decoder network using LSTM in Keras but I want to have a ConvLSTM encoder-decoder and since the ConvLSTM2D does not accept any 'initial_state' argument so I can pass the initial state of the encoder to the decoder, I tried to use RNN in Keras and tried to pass the ConvLSTM2D as the cell of RNN but I got the following error:

ValueError: ('`cell` should have a `call` method. The RNN was passed:', <tf.Tensor 'encoder_1/TensorArrayReadV3:0' shape=(?, 62, 62, 32) dtype=float32>)

This is how I tried to define the RNN cell:

first_input = Input(shape=(None, 62, 62, 12))
encoder_convlstm2d = ConvLSTM2D(filters=32, kernel_size=(3, 3),
                                    padding='same',
                                    name='encoder'+ str(1))(first_input )
encoder_outputs, state_h, state_c = keras.layers.RNN(cell=encoder_convlstm2d, return_sequences=False, return_state=True, go_backwards=False,
                 stateful=False, unroll=False)
PrinceZee
  • 1,587
  • 3
  • 11
  • 24
MRM
  • 1,099
  • 2
  • 12
  • 29

1 Answers1

0

Below is my approach for encoder-decoder-based solution with ConvLSTM.

def convlstm(input_shape):

    print(np.shape(input_shape))

    inpTensor = Input((input_shape))

    #encoder
    net1 = ConvLSTM2D(filters=32, kernel_size=3,
                   padding='same', return_sequences=True)(inpTensor)

    max_pool1 = MaxPooling3D(pool_size=(2, 2, 2), strides=2, 
    padding='same')(net1)

    bn1 = BatchNormalization(axis=1)(max_pool1)

    dp1 = Dropout(0.2)(bn1)

    net2 = ConvLSTM2D(filters=64, kernel_size=3,
                    padding='same', return_sequences=True)(dp1)

    max_pool2 = MaxPooling3D(pool_size=(2, 2, 2), strides=2, 
    padding='same')(net2)

    bn2 = BatchNormalization(axis=1)(max_pool2)

    dp2 = Dropout(0.2)(bn2)

    net3 = ConvLSTM2D(filters=128, kernel_size=3,
                   padding='same', return_sequences=True)(dp2)

    max_pool3 = MaxPooling3D(pool_size=(2, 2, 2), strides=2, 
    padding='same')(net3)

    bn3 = BatchNormalization(axis=1)(max_pool3)

    dp3 = Dropout(0.2)(bn3)


    #decoder
    net4 = ConvLSTM2D(filters=128, kernel_size=3,
                    padding='same', return_sequences=True)(dp3)

    up1 = UpSampling3D((2, 2, 2))(net4)

    net5= ConvLSTM2D(filters=64, kernel_size=3,
                    padding='same', return_sequences=True)(up1)

    up2 = UpSampling3D((2, 2, 2))(net5)

    net6 = ConvLSTM2D(filters=32, kernel_size=3,
                    padding='same', return_sequences=False)(up2)

    up3 = UpSampling2D((2, 2))(net6)

    out = Conv2D(filters=1, kernel_size=(3, 3), activation='sigmoid',
                  padding='same', data_format='channels_last')(up3)

    #or use only return out
    return Model(inpTensor, out)
Saeed Ullah
  • 302
  • 3
  • 11