1

I need to use a pre-trained model in Keras(keras.applications.VGG16) as a baseline for creating another model(for doing transfer learning) from the first layers of it. The end goal is to freeze and export the model for deployment on a raspberry pi with AIY vision kit.

I've tried the common approach of:

model_base = keras.applications.VGG16(input_tensor=inputs)
x = model_base.get_layer(backbone_layer).output
x = keras.layers.Flatten()(x)
x = keras.layers.Dense(1)(x)
model = keras.models.Model(inputs=inputs, outputs=x)

I'm also trying to use :

model_base._layers.pop()

I call pop() n times where "n" is the number of final layers i want to get rid of.

And it seems to work in both cases, when i use new_model.summary() it only shows the desired first layers of the VGG16 model plus the new layers added for customization, however when exporting the model and compiling it for tf-lite the compiler returns:

Not enough on-device memory to run model

Which is weird since the resulting model is even smaller (the .pb file and numer of layers) than other models manually defined that can be correctly imported, after analyzing tensorboard and exporting the .pb file as text i found that the original model is being exported too( all the layers even the ones not used and removed with pop()) not just the new one

(as seen in tensorboard there are 2 parallel models, in the right the desired one, but the original layers of the original model are still shown in the left, also that part of the original model is present in the exported .pb file)

enter image description here

My question is how can i definitely remove the unused layers from the keras.applications.VGG16 model and just keep the first layers + new custom layers ? using pop() has not worked, also tried del layer (in a for loop) unsuccessfully.

Or what other alternatives do i have in order to use a pre-trained model as baseline by keeping only it's first layers and then connecting it to some other custom layers.

Luis Leal
  • 3,388
  • 5
  • 26
  • 49

3 Answers3

2

The easiest way is in VGG16 set include_top=False, pooling=max. Then see code below:

x=backbone.layers[-1].output
predictions=Dense (len(classes), activation='softmax')(x)
model = Model(inputs=backbone.input, outputs=predictions)    
model.compile(Adamax(lr=lr_rate), loss='categorical_crossentropy', metrics=['accuracy'])
Gerry P
  • 7,662
  • 3
  • 10
  • 20
  • Thanks for your answer, i tried that too, but the question is not how to do it, but how to make sure the original model is not exported when doing it, i updated the question to make it clearer – Luis Leal Aug 01 '20 at 23:05
0

It's not actually remove the layer(s) but rather create a new model without it.

I don't really get what you want to do but object detectors usually do this.

inputs = keras.layers.Input((None, None, 3), include_top=False)
backbone = keras.applications.VGG16(input_tensor=inputs)
x = backbone.get_layer('block1_conv2').output
x = keras.layers.Dense(50)(x)
model = keras.models.Model(inputs=inputs, outputs=x, name=backbone.name)

This will result in a model with only the first two layers and add a new Dense as output.

Natthaphon Hongcharoen
  • 2,244
  • 1
  • 9
  • 23
  • Thanks for your answer, i edited the question to be more clear, basically that one i also tried and the issue is that the complete graph is still shown in tensorboard and exported when i freeze the graph, even the undesired layers. – Luis Leal Aug 01 '20 at 21:11
  • Ok now I understand your question. Results seem strange to me as well. How did you export the model? Did you use model.save? – Gerry P Aug 02 '20 at 04:31
  • 1
    @Luis `model.save` and load only the edited model should work though – Natthaphon Hongcharoen Aug 02 '20 at 06:19
  • @NatthaphonHongcharoen saving and loading worked (i added a clear session call but the main idea was saving and loading) – Luis Leal Aug 08 '20 at 05:56
0

Saving the desired with model.save() , clearing tf session and then loading it again fixed the issue(as suggested @NatthaphonHongcharoen (in the comments):

model.save(model_file)
del model
keras.backend.clear_session()
model = keras.models.load_model(model_file)

Now the exported graph in tensorflow shows only the desired layers and the frozen graph generates a smaller .pb file.

However another issue persists, even using a single layer of the base model and adding a Dense layer the compiler still says

Not enough on-device memory to run model.

But that is a different issue not directly asked in this question.

Luis Leal
  • 3,388
  • 5
  • 26
  • 49