1

Stealing the example code from here:

def custom_loss_wrapper(input_tensor):
    def custom_loss(y_true, y_pred):
        return K.binary_crossentropy(y_true, y_pred) + K.mean(input_tensor)
    return custom_loss

input_tensor = Input(shape=(10,))
hidden = Dense(100, activation='relu')(input_tensor)
out = Dense(1, activation='sigmoid')(hidden)
model = Model(input_tensor, out)
model.compile(loss=custom_loss_wrapper(input_tensor), optimizer='adam')

X = np.random.rand(1000, 10)
y = np.random.rand(1000, 1)
model.train_on_batch(X, y)

This no longer works in recent Tensorflow versions. The main solution I have seen is to disable eager execution, but this will break other things I'm doing.

How can this type of functionality be maintained while staying in eager mode? That is, passing an input to the loss function? The best I can think of is to modify the network to concatenate the input to the output, then pull it apart in the loss. Very kludgy though.

Marco Cerliani
  • 21,233
  • 3
  • 49
  • 54
Mastiff
  • 2,101
  • 2
  • 23
  • 38

1 Answers1

3

You can use add_loss to pass external layers to your loss, in your case the input tensor.

Here an example:

def CustomLoss(y_true, y_pred, input_tensor):
    return K.binary_crossentropy(y_true, y_pred) + K.mean(input_tensor)

X = np.random.uniform(0,1, (1000,10))
y = np.random.randint(0,2, (1000,1))

inp = Input(shape=(10,))
hidden = Dense(100, activation='relu')(inp)
out = Dense(1, activation='sigmoid')(hidden)
target = Input((1,))
model = Model([inp,target], out)

model.add_loss( CustomLoss( target, out, inp ) )
model.compile(loss=None, optimizer='adam')
model.fit(x=[X,y], y=None, epochs=3)

To use the model in inference mode (removing the target from inputs)

final_model = Model(model.input[0], model.output)
final_model.predict(X)
Marco Cerliani
  • 21,233
  • 3
  • 49
  • 54