23

I am trying to create the custom loss function using Keras. I want to compute the loss function based on the input and predicted the output of the neural network.

I tried using the customloss function in Keras. I think y_true is the output that we give for training and y_pred is the predicted output of the neural network. The below loss function is same as "mean_squared_error" loss in Keras. 

def customloss(y_true, y_pred):
    return K.mean(K.square(y_pred - y_true), axis=-1)

I would like to use the input to the neural network also to compute the custom loss function in addition to mean_squared_error loss. Is there a way to send an input to the neural network as an argument to the customloss function.

Thank you.

user3443033
  • 737
  • 2
  • 6
  • 21

2 Answers2

38

I have come across 2 solutions to the question you asked.

  1. You can pass your input (scalar only) as an argument to the custom loss wrapper function.
    def custom_loss(i):

        def loss(y_true, y_pred):
            return K.mean(K.square(y_pred - y_true), axis=-1) + something with i...
        return loss

    def baseline_model():
        # create model
        i = Input(shape=(5,))
        x = Dense(5, kernel_initializer='glorot_uniform', activation='linear')(i)
        o = Dense(1, kernel_initializer='normal', activation='linear')(x)
        model = Model(i, o)
        model.compile(loss=custom_loss(i), optimizer=Adam(lr=0.0005))
        return model

This solution is also mentioned in the accepted answer here

  1. You can pad your label with extra data columns from input and write a custom loss. This is helpful if you just want one/few feature column(s) from your input.
    def custom_loss(data, y_pred):

        y_true = data[:, 0]
        i = data[:, 1]
        return K.mean(K.square(y_pred - y_true), axis=-1) + something with i...


    def baseline_model():
        # create model
        i = Input(shape=(5,))
        x = Dense(5, kernel_initializer='glorot_uniform', activation='linear')(i)
        o = Dense(1, kernel_initializer='normal', activation='linear')(x)
        model = Model(i, o)
        model.compile(loss=custom_loss, optimizer=Adam(lr=0.0005))
        return model


    model.fit(X, np.append(Y_true, X[:, 0], axis =1), batch_size = batch_size, epochs=90, shuffle=True, verbose=1)

This solution can be found also here in this thread.

I have only used the 2nd method when I had to use input feature columns in the loss. The first method can be only used with scalar arguments as mentioned in the comments.

Anakin
  • 1,889
  • 1
  • 13
  • 27
  • 2
    I'm very late to the party, but your second solution is just genius! Very very practical, awesome. – Jean-Pierre Coffe May 03 '20 at 18:18
  • 2
    Doing so I am getting: `tensorflow.python.eager.core._SymbolicException: Inputs to eager execution function cannot be Keras symbolic tensors, but found []`. Not that I am passing a `tf.data.Dataset` to the `fit()` function. Any idea what the problem is here? – Stefan Falk Jul 02 '20 at 08:12
  • I am having trouble using the first method. I get this error `UnboundLocalError: local variable 'input_tensor' referenced before assignment` – corvo Sep 18 '20 at 12:33
  • 1
    In the 2nd method, 2 columns are passed as y for model.fit(x, y). If we make predictions using the trained model, what column does it predict? @Anakin – JayaniH Dec 07 '20 at 09:28
  • I've just implemented your 2nd solution and it works like a charm ! I concur with Jean-Pierre - 'just genius'. Thx – Toren Feb 13 '21 at 23:47
  • 1
    @Anakin The 1st method does **not** work with tensor input, like, for example, input layer of the model. – Marcin Wojnarski Jan 26 '23 at 12:28
5

You could wrap your custom loss with another function that takes the input tensor as an argument:

def customloss(x):
    def loss(y_true, y_pred):
        # Use x here as you wish
        err = K.mean(K.square(y_pred - y_true), axis=-1)
        return err

    return loss

And then compile your model as follows:

model.compile('sgd', customloss(x))

where x is your input tensor.

NOTE: Not tested.

rvinas
  • 11,824
  • 36
  • 58
  • What is y_true and y_pred? Is it the tensor or just the scalar for one input? I want to send an input x corresponding to y_true. Is the loss calculated individually for each element of ytrain tensor (output of the Keras model) or for the whole tensor? Thank you. – user3443033 Apr 01 '19 at 14:49
  • `y_true` is the label's tensor, while `y_pred` is the network prediction's tensor. As for the other question, I don't understand what is being asked. – rvinas Apr 01 '19 at 22:01
  • y_pred will be for certain input to the model. I would like to access that input tensor. Is there any way to send an input tensor to the customloss function? thank you. – user3443033 Apr 02 '19 at 00:09
  • You can pass the input tensor via the argument `x` of `customloss` – rvinas Apr 02 '19 at 08:16