Keras training only specific outputs

Question

I am using Kears with tensorflow and I have a model with 3 output out of which I only want to train 2.

model = Model(input=input, output=[out1,out2,out3])
model.compile(loss=[loss1, loss2, loss3], optimizer=my_optimizer)

loss1(y_true, y_pred):
    return calculate_loss1(y_true, y_pred)

loss2(y_true, y_pred):
    return calculate_loss2(y_true, y_pred)

loss3(y_true, y_pred):
    return 0.0*K.mean(y_pred)

I tried to do it with the code above but I am not sure it does what I want do do. So I think it adds up the losses and it trains each output with that loss meanwhile I do not wish to train out3 at all. (I need out3 because it is used in testing). Could anybody tell me how to achieve this or reassure me that the code actually dose what I want?

score 23 · Accepted Answer · answered Mar 14 '17 at 12:08

23

You have to create 2 different models like this

model1 = Model(input=input, output=[out1,out2])
model2 = Model(input=input, output=[out1,out2,out3])

You compile both but only fit the first. They will share the layers so model2, even if it wasn't trained, will have the weights learned from model1. But if there is a layer in out3 which is trainable but not in the flow between input and out1 and out2 of the graph, that layer wont be trained so will stay wirh its inital values.

Does that help? :-)

answered Mar 14 '17 at 12:08

Nassim Ben

11,473
1
34
52

Yes it does help! Thank you very much :) – DalekSupreme Mar 14 '17 at 12:10
Great :-) i'm glad – Nassim Ben Mar 14 '17 at 12:11
Sorry just one more thing. Will it work with model2 = Model(input=input, output=[out3]) as well? – DalekSupreme Mar 14 '17 at 13:20
2

Sure, what's happenning when you create the model object it's compiling all the layers in the graph that go from input to output nodes... if in between there are layers that you trained in another model, their weights will be trained. Thats the power of the functionnal API and layers/model objects :-) – Nassim Ben Mar 14 '17 at 13:23
Thanks you for your help ;) – DalekSupreme Mar 14 '17 at 13:27
I read this source: https://stackoverflow.com/questions/40446488/training-only-one-output-of-a-network-in-keras. Which one is more appropriate? – donto Dec 28 '19 at 08:33
@donto they do basically the same thing – Bersan May 09 '20 at 20:12
@Nassim Ben How do we know that the models will have the same weights? As we are creating two different models, we can have two different copies of weights right. – omsrisagar Oct 27 '20 at 23:39
You are a genius. Thank you so much! – krenerd Feb 22 '21 at 04:52

score 1 · Answer 2 · answered Nov 24 '20 at 22:00

You can set one of the losses to None:

model = Model(input=input, output=[out1,out2,out3])
model.compile(loss=[loss1, loss2, None], optimizer=my_optimizer)


loss1(y_true, y_pred):
    return calculate_loss1(y_true, y_pred)


loss2(y_true, y_pred):
    return calculate_loss2(y_true, y_pred)

Keras training only specific outputs

2 Answers2

Linked