Keras replacing input layer

Question

The code that I have (that I can't change) uses the Resnet with my_input_tensor as the input_tensor.

model1 = keras.applications.resnet50.ResNet50(input_tensor=my_input_tensor, weights='imagenet')

Investigating the source code, ResNet50 function creates a new keras Input Layer with my_input_tensor and then create the rest of the model. This is the behavior that I want to copy with my own model. I load my model from h5 file.

model2 = keras.models.load_model('my_model.h5')

Since this model already has an Input Layer, I want to replace it with a new Input Layer defined with my_input_tensor.

How can I replace an input layer?

I haven't. I looked up the documentation. Maybe `model2(my_input_tensor)` can be somehow used? — zcadqe, Mar 29 '18 at 04:01
If the model is not sequential (as i can see its resnet50) the solution is below, if it is, you can use `model.add()`. — Milind Deore, Mar 29 '18 at 11:08

score 28 · Accepted Answer · answered Mar 29 '18 at 11:04

28

When you saved your model using:

old_model.save('my_model.h5')

it will save following:

The architecture of the model, allowing to create the model.
The weights of the model.
The training configuration of the model (loss, optimizer).
The state of the optimizer, allowing training to resume from where you left before.

So then, when you load the model:

res50_model = load_model('my_model.h5')

you should get the same model back, you can verify the same using:

res50_model.summary()
res50_model.get_weights()

Now you can, pop the input layer and add your own using:

res50_model.layers.pop(0)
res50_model.summary()

add new input layer:

newInput = Input(batch_shape=(0,299,299,3))    # let us say this new InputLayer
newOutputs = res50_model(newInput)
newModel = Model(newInput, newOutputs)

newModel.summary()
res50_model.summary()

answered Mar 29 '18 at 11:04

Milind Deore

2,887
5
25
40

4

Since I have input tensor that I want to use, I ended using. `new_input_layer = keras.layers.Input(tensor=my_input_tensor)` and followed your suggestion. It works! Thanks. – zcadqe Mar 30 '18 at 01:52
6

It seems odd that after the `model.layers.pop(0)` call that `model.input` still shows original input layer. I was trying the posted solution with VGG16 model. – user3731622 Mar 15 '19 at 19:35
1

THIS WILL NOT WORK, if we create a resnet18, then we remove the first layer (pop) then we create a model with another input and then the resnet, what happens is that it creates a model with one input layer and the second a "model" layer, which is disconnected from the other. Plus, this also denies the possibility of concatenating/merging with other models (this is mainly why I am complaining). – dberga Dec 17 '19 at 12:15
2

`model.layers.pop(0)` won't change anything. https://github.com/tensorflow/tensorflow/issues/22479 – Jason Sep 06 '20 at 06:38

ben26941 · Answer 2 · 2019-10-21T14:17:52.980

Layers.pop(0) or anything like that doesn't work.

You have two options that you can try:

1.

You can create a new model with the required layers.

A relatively easy way to do this is to i) extract the model json configuration, ii) change it appropriately, iii) create a new model from it, and then iv) copy over the weights. I'll just show the basic idea.

i) extract the configuration

model_config = model.get_config()

ii) change the configuration

input_layer_name = model_config['layers'][0]['name']
model_config['layers'][0] = {
                      'name': 'new_input',
                      'class_name': 'InputLayer',
                      'config': {
                          'batch_input_shape': (None, 300, 300),
                          'dtype': 'float32',
                          'sparse': False,
                          'name': 'new_input'
                      },
                      'inbound_nodes': []
                  }
model_config['layers'][1]['inbound_nodes'] = [[['new_input', 0, 0, {}]]]
model_config['input_layers'] = [['new_input', 0, 0]]

ii) create a new model

new_model = model.__class__.from_config(model_config, custom_objects={})  # change custom objects if necessary

ii) copy weights

# iterate over all the layers that we want to get weights from
weights = [layer.get_weights() for layer in model.layers[1:]]
for layer, weight in zip(new_model.layers[1:], weights):
    layer.set_weights(weight)

2.

You can try a library like kerassurgeon (I am linking to a fork that works with the tensorflow keras version). Note that insertion and deletion operations only work under certain conditions such as compatible dimensions.

from kerassurgeon.operations import delete_layer, insert_layer

model = delete_layer(model, layer_1)
# insert new_layer_1 before layer_2 in a model
model = insert_layer(model, layer_2, new_layer_3)

This is actually the best solution I've found. Thanks for sharing it. — Ansel D'souza, Mar 15 '22 at 19:16

score 9 · Answer 3 · answered Mar 13 '19 at 16:14

The solution from @MilindDeore did not work for me, unfortunately. While I can print the summary of the new model, I receive a "Matrix size incompatible" error upon prediction. I guess this makes sense, since the new input shape of the dense layer does not match the shape of the old dense layer weights.

Thus, here is another solution. The key for me was to use "_layers" instead of "layers". The latter only seems to return a copy.

import keras
import numpy as np

def get_model():
    old_input_shape = (20, 20, 3)
    model = keras.models.Sequential()
    model.add(keras.layers.Conv2D(9, (3, 3), padding="same", input_shape=old_input_shape))
    model.add(keras.layers.MaxPooling2D((2, 2)))
    model.add(keras.layers.Flatten())
    model.add(keras.layers.Dense(1, activation="sigmoid"))
    model.compile(loss='binary_crossentropy', optimizer=keras.optimizers.Adam(lr=0.0001), metrics=['acc'], )
    model.summary()
    return model

def change_model(model, new_input_shape=(None, 40, 40, 3)):
    # replace input shape of first layer
    model._layers[1].batch_input_shape = new_input_shape

    # feel free to modify additional parameters of other layers, for example...
    model._layers[2].pool_size = (8, 8)
    model._layers[2].strides = (8, 8)

    # rebuild model architecture by exporting and importing via json
    new_model = keras.models.model_from_json(model.to_json())
    new_model.summary()

    # copy weights from old model to new one
    for layer in new_model.layers:
        try:
            layer.set_weights(model.get_layer(name=layer.name).get_weights())
        except:
            print("Could not transfer weights for layer {}".format(layer.name))

    # test new model on a random input image
    X = np.random.rand(10, 40, 40, 3)
    y_pred = new_model.predict(X)
    print(y_pred)

    return new_model

if __name__ == '__main__':
    model = get_model()
    new_model = change_model(model)

Incase you are using tf.keras from 2.X, then use `layer._batch_input_size` with starting underscore. — Sreekant Shenoy, May 27 '21 at 15:16

score 4 · Answer 4 · answered Dec 03 '19 at 22:01

4

Unfortunately kerassurgeon did not support my model as I had frozen layers. I had to make a small change to @MilindDeore's solution - replace model.layers.pop(0) to model._layers.pop(0) and it worked for me. Note that I am using tf.keras in TF 2.0.

answered Dec 03 '19 at 22:01

Karthikeyan Sv

61
3

score 4 · Answer 5 · edited Apr 13 '20 at 20:53

4

For tf.keras in Tensorflow 2, using tfsurgeon did not work as I had custom layers.

What did work was changing layer._batch_input_size (note the underscore)

Hacky, but got the job done.

edited Apr 13 '20 at 20:53

M--

25,431
8
61
93

answered Apr 13 '20 at 18:46

velochy

371
2
12

score 3 · Answer 6 · answered Oct 24 '19 at 20:59

This should be pretty easy with kerassurgeon. First you need to install the library; depending on if you are using Keras through TensorFlow (with tf 2.0 and up) or Keras as a separate library, it needs to be installed in different ways.

For Keras in TF: pip install tfkerassurgeon (https://github.com/Raukk/tf-keras-surgeon). For standalone Keras: pip install kerassurgeon (https://github.com/BenWhetton/keras-surgeon)

To replace the input (example with TF 2.0; currently untested code):

from tensorflow import keras  # or import keras for standalone version
from tensorflow.keras.layers import Input

model = keras.models.load_model('my_model.h5')
my_input_tensor = Input(input_shape=(260, 260, 3))

# or kerassurgeon for standalone Keras
from tfkerassurgeon import delete_layer, insert_layer

model = delete_layer(model.layers[0])
# inserts before layer 0
model = insert_layer(model.layers[0], my_input_tensor)

To update this answer: "tfkerassurgeon" is no longer being maintained and does not work for the current TensorFlow version (2.5.0). On the other hand, "kerassurgeon" DOES support Keras through TensorFlow (i.e tensorflow.keras), not just standalone Keras. — SMD01, Jul 23 '21 at 16:03

score 0 · Answer 7 · answered Apr 18 '23 at 10:30

For TensorFlow 2, we can modify batch_input_shape:

model.summary()
model_config = model.get_config()
model_config["layers"][0]["config"]["batch_input_shape"] = (None, 10, 1)
modified_model = tf.keras.Model.from_config(model_config)
modified_model.summary()

Keras replacing input layer

7 Answers7

Linked