How do I split an convolutional autoencoder?

Question

I have compiled an autoencoder (full code is below), and after training it I would like to split it into two separate models: encoder (layers e1...encoded) and decoder (all other layers) in which to feed manually modified images that had been encoded by the decoder. I have succeeded in creating an encoder as a separate model with:

encoder = Model(input_img, autoencoder.layers[6].output)

But the same approach fails when I try to make a decoder:

encoded_input = Input(shape=(4,4,8))
decoder = Model(input_img, decoded)

This is my full code:

from keras.layers import Input, Dense, Conv2D, MaxPooling2D, UpSampling2D
from keras.models import Model
from keras import backend as K
from keras.models import load_model

input_img = Input(shape=(28, 28, 1))  # adapt this if using channels_first` image data format

e1 = Conv2D(16, (3, 3), activation='relu', padding='same')(input_img)
e2 = MaxPooling2D((2, 2), padding='same')(e1)
e3 = Conv2D(8, (3, 3), activation='relu', padding='same')(e2)
e4 = MaxPooling2D((2, 2), padding='same')(e3)
e5 = Conv2D(8, (3, 3), activation='relu', padding='same')(e4)
encoded = MaxPooling2D((2, 2), padding='same')(e5)

# at this point the representation is (4, 4, 8) i.e. 128-dimensional

d1 = Conv2D(8, (3, 3), activation='relu', padding='same')(encoded)
d2 = UpSampling2D((2, 2))(d1)
d3 = Conv2D(8, (3, 3), activation='relu', padding='same')(d2)
d4 = UpSampling2D((2, 2))(d3)
d5 = Conv2D(16, (3, 3), activation='relu')(d4)
d6 = UpSampling2D((2, 2))(d5)
decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(d6)

autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

Plese help.

EDIT By the way, I was able to do this with an autoencoder consisting of dense layers:

from keras.layers import Input, Dense
from keras.models import Model

# this is the size of our encoded representations
encoding_dim = 32  # 32 floats -> compression of factor 24.5, assuming     the input is 784 floats

# this is our input placeholder
input_img = Input(shape=(784,))

# "encoded" is the encoded representation of the input
encoded = Dense(encoding_dim, activation='relu')(input_img)

# "decoded" is the lossy reconstruction of the input
decoded = Dense(784, activation='sigmoid')(encoded)

# this model maps an input to its reconstruction
autoencoder = Model(input_img, decoded)

# this model maps an input to its encoded representation
encoder = Model(input_img, encoded)

# create a placeholder for an encoded (32-dimensional) input
encoded_input = Input(shape=(encoding_dim,))

# retrieve the last layer of the autoencoder model
decoder_layer = autoencoder.layers[-1]

# create the decoder model
decoder = Model(encoded_input, decoder_layer(encoded_input))

Try to follow the idea shown in this [answer](https://stackoverflow.com/questions/44371383/valueerror-input-0-is-incompatible-with-layer-dense-6-in-keras/44467226#44467226). — Marcin Możejko, Feb 04 '18 at 18:06

score 7 · Accepted Answer · answered Feb 04 '18 at 19:57

Ok, I figured this out after a few hours. What worked for me was to: 1. Create a separate model for the encoder 2. Create a separate model for teh decoder 3. Create a general model for the autoencoder:

autoencoder = Model(input, Decoder()(Encoder(input))

The full working code is below:

def Encoder():
    input_img = Input(shape=(28, 28, 1))  # adapt this if using `channels_first` image data format   
    e1 = Conv2D(16, (3, 3), activation='relu', padding='same')(input_img)
    e2 = MaxPooling2D((2, 2), padding='same')(e1)
    e3 = Conv2D(8, (3, 3), activation='relu', padding='same')(e2)
    e4 = MaxPooling2D((2, 2), padding='same')(e3)
    e5 = Conv2D(8, (3, 3), activation='relu', padding='same')(e4)
    e6 = MaxPooling2D((2, 2), padding='same')(e5)
    return Model(input_img, e6)


def Decoder():
    input_img = Input(shape=(4, 4, 8))  # adapt this if using `channels_first` image data format   
    d1 = Conv2D(8, (3, 3), activation='relu', padding='same')(input_img)
    d2 = UpSampling2D((2, 2))(d1)
    d3 = Conv2D(8, (3, 3), activation='relu', padding='same')(d2)
    d4 = UpSampling2D((2, 2))(d3)
    d5 = Conv2D(16, (3, 3), activation='relu')(d4)
    d6 = UpSampling2D((2, 2))(d5)
    d7 = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(d6)
    return Model(input_img, d7)


# define input to the model:
x = Input(shape=(28, 28, 1))

# make the model:
autoencoder = Model(x, Decoder()(Encoder()(x)))

# compile the model:
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

this works but the real question is, how do you extract the encoder/decoder components **after** the autoencoder has been trained — pcko1, Jan 13 '19 at 17:43
@pcko1 This has been a while, but I think you can at least get the layer weights using get_weights(), or you can make `autoencoder = Model(x, d(e(x)))` where `d = Decoder()` and `e = Encoder()`. After you train the autoencoder, you can use `d` and `e` separately. — MegaNightdude, Jan 15 '19 at 07:41
yes, correct! I was just wondering whether it could be possible to train an arbitrarily complex network and then split it into two sub-models, without having explicitly defined the network as a Model of two sub-models before training. No worries though, your scheme works for me too :) — pcko1, Jan 16 '19 at 18:31

How do I split an convolutional autoencoder?

1 Answers1

Linked