Make a deep copy of a keras model in python

Question

I would like to make a deep copy of a keras model (called model1) of mine in order to be able to use it in a for a loop and then re-initialize for each for-loop iteration and perform fit with one additional sample to the model. I would like to be able to initialize the model after each iteration since after performing the fit (my model is modified however, I want it keep it as it is when i am loading from the path using the load_weights).

My code looks like:

model1= create_Model()
model1.compile(optimizer='rmsprop', loss='categorical_crossentropy')
model1.load_weights('my_weights')

model_copy= create_Model()
model_copy.compile(optimizer='rmsprop', loss='categorical_crossentropy')

model_copy= keras.models.clone_model(model1)
for j in range(0, image_size):
      model_copy.fit(sample[j], sample_lbl[j])
      prediction= model_copy.predict(sample[j])

Also, it is not really efficient for me to load the model each time in the for-loop since that is time-consuming. How can I do properly the deep copy in my case? The code I posted give the following error that concerns the function .fit and my reference model model_copy:

RuntimeError: You must compile a model before training/testing. Use model.compile(optimizer, loss).

Answer 1

The issue is that model_copy is probably not compiled after cloning. There are in fact a few issues:

1. Apparently cloning doesn't copy over the loss function, optimizer info, etc.

2. Before compiling you need to also build the model.

3. Moreover, cloning doesn't copy weight over

So you need a couple extra lines after cloning. For example, for 10 input variables:

model_copy= keras.models.clone_model(model1)
model_copy.build((None, 10)) # replace 10 with number of variables in input layer
model_copy.compile(optimizer='rmsprop', loss='categorical_crossentropy')
model_copy.set_weights(model.get_weights())

---

Easier Method 1: Loading weights from file

If I understand your question correctly, there is an easier way to do this. You don't need to clone the model, just need to save the old_weights and set the weights at beginning of the loop. You can simply load weights from file as you are doing.

for _ in range(10):
    model1= create_Model()
    model1.compile(optimizer='rmsprop', loss='categorical_crossentropy')
    model1.load_weights('my_weights')

    for j in range(0, image_size):
          model1.fit(sample[j], sample_lbl[j])
          prediction= model1.predict(sample[j])

---

Easier Method 2: Loading weights from previous get_weights()

Or if you prefer not to load from file:

model1= create_Model()
model1.compile(optimizer='rmsprop', loss='categorical_crossentropy')
model1.load_weights('my_weights')
old_weights = model1.get_weights()

for _ in range(10):
    model1.set_weights(old_weights)
    for j in range(0, image_size):
          model1.fit(sample[j], sample_lbl[j])
          prediction= model1.predict(sample[j])

Answer 2

These days it's trivial:

model2 = tf.keras.models.clone_model(model1)

This will give you a new model, new layers, and new weights. If for some reason that doesn't work (I haven't tested it) this older solution will:

model1 = Model(...)
model1.compile(...)
model1.save(savepath) # saves compiled state
model2 = keras.models.load_model(savepath)

Answer 3

A very general method to get deep copies in python is deepcopy from the copy package:

import copy
model2=copy.deepcopy(model)

Are there any disadvantages in using this for keras models?

Edit: As pointed out in the comment by GRASBOCK, this solution (copy.deepcopy) does not work reliably for tf-models: https://stackoverflow.com/a/64427748/5044463.