Removing rescaling layer from a functional model

Question

Is there a way to remove a rescaling layer from a functional model?

I'm trying to remove the rescaling layer from this model:

efficientnetb0_model.layers[0:10]

Output:

[<tensorflow.python.keras.engine.input_layer.InputLayer at 0x18d5e873d90>,
 <tensorflow.python.keras.layers.preprocessing.image_preprocessing.Rescaling at 0x18d5e8a2d00>,
 <tensorflow.python.keras.layers.preprocessing.normalization.Normalization at 0x18d5eb48100>,
 <tensorflow.python.keras.layers.convolutional.ZeroPadding2D at 0x18d5eb48550>,
 <tensorflow.python.keras.layers.convolutional.Conv2D at 0x18d5eb7f670>,
 <tensorflow.python.keras.layers.normalization_v2.BatchNormalization at 0x18d5eba2c70>,
 <tensorflow.python.keras.layers.core.Activation at 0x18d5eb481f0>,
 <tensorflow.python.keras.layers.convolutional.DepthwiseConv2D at 0x18d5ebe8e50>,
 <tensorflow.python.keras.layers.normalization_v2.BatchNormalization at 0x18d5ec08e20>,
 <tensorflow.python.keras.layers.core.Activation at 0x18d5ec2f1c0>]

I tried this:

efficientnetb0_model.layers.pop(1)
efficientnetb0_model.layers[0:10]

But nothing happens:

[<tensorflow.python.keras.engine.input_layer.InputLayer at 0x18d5e873d90>,
 <tensorflow.python.keras.layers.preprocessing.image_preprocessing.Rescaling at 0x18d5e8a2d00>,
 <tensorflow.python.keras.layers.preprocessing.normalization.Normalization at 0x18d5eb48100>,
 <tensorflow.python.keras.layers.convolutional.ZeroPadding2D at 0x18d5eb48550>,
 <tensorflow.python.keras.layers.convolutional.Conv2D at 0x18d5eb7f670>,
 <tensorflow.python.keras.layers.normalization_v2.BatchNormalization at 0x18d5eba2c70>,
 <tensorflow.python.keras.layers.core.Activation at 0x18d5eb481f0>,
 <tensorflow.python.keras.layers.convolutional.DepthwiseConv2D at 0x18d5ebe8e50>,
 <tensorflow.python.keras.layers.normalization_v2.BatchNormalization at 0x18d5ec08e20>,
 <tensorflow.python.keras.layers.core.Activation at 0x18d5ec2f1c0>]

Does this answer your question? [How to add and remove new layers in keras after loading weights?](https://stackoverflow.com/questions/41668813/how-to-add-and-remove-new-layers-in-keras-after-loading-weights) — Dominik Ficek, Jul 17 '21 at 14:16
@DominikFicek unfortunately, no. If my understanding is right, that question answers adding and removing the last layers. Mine is about removing intermediate layers. — djbacs, Jul 17 '21 at 14:23

score 2 · Answer 1 · answered Jul 18 '21 at 11:02

I could not find a way to remove the intermediate Rescaling layer. But, by modifying the scale parameter of the Rescaling layer, we can nullify the transformation made by the layer.

This can be done by setting scale to 1. See the docs for the scale argument.

import tensorflow as tf

model = tf.keras.applications.EfficientNetB0( weights='imagenet' , include_top=False )
print( model.layers )

# 2nd layer in EfficientNetB0
print( model.layers[ 1 ].scale )

The output is,

0.00392156862745098

By setting the scale parameter to 1.0,

model.layers[ 1 ].scale = 1.0
print( model.layers[ 1 ].scale )

Removing rescaling layer from a functional model

1 Answers1