0

I want to create a custom loss function for a Keras deep learning regression model. For the custom loss function, I want to use a feature that is in the dataset but I am not using that particular feature as an input to the model.

My data looks like this:

X  |  Y  | feature
---|-----|--------
x1 | y1  | f1
x2 | y2  | f2

The input to the model is X and I want to predict Y using the model. I want something like the following as the loss function:

def custom_loss(feature):
    def loss(y_true, y_pred):
        root_mean__square(y_true - y_pred) + std(y_pred - feature)
    return loss

I can't use a wrapper function as above, because the feature values depends on the training and test batches, thus cannot be passed to the custom loss function at the model compile time. How can I use the additional feature in the dataset to create a custom loss function?

EDIT:

I did the following based on an answer on this thread. When I make predictions using this model, does it make predictions for 'Y' or a combination of Y and the additional feature? I want to make sure because model.fit( ) takes both 'Y' and 'feature' as y to train but model.predict( ) only gives the one output. If the predictions are a combination of Y and the additional feature, how can I extract only Y?

    def custom_loss(data, y_pred):

        y_true = data[:, 0]
        feature = data[:, 1]
        return K.mean(K.square((y_pred - y_true) + K.std(y__pred - feature)))

    def create_model():
        # create model
        model = Sequential()
        model.add(Dense(5, input_dim=1, activation="relu"))
        model.add(Dense(1, activation="linear"))

    (train, test) = train_test_split(df, test_size=0.3, random_state=42)

    model = models.create_model(train["X"].shape[1])
    opt = Adam(learning_rate=1e-2, decay=1e-3/200)
    model.compile(loss=custom_loss, optimizer=opt)


    model.fit(train["X"], train[["Y", "feature"]], validation_data=(test["X"], test[["Y", "feature"]]), batch_size = 8, epochs=90)

    predY = model.predict(test["X"]) # what does the model predict here?
JayaniH
  • 293
  • 1
  • 2
  • 9
  • 1
    You can pass the `feature` as a separate input to your model with an `Input` layer. Pass this tensor in the `outputs=` argument while initializing the `Model( inputs, outputs )`. So, `y_pred` will consist of the actual prediction of the model as well as the extra `feature` as a tensor. – Shubham Panchal Dec 07 '20 at 08:28
  • @ShubhamPanchal How do I access the feature in the custom loss function? – JayaniH Dec 07 '20 at 09:26

2 Answers2

0

First check the data structure of your input Y in fit function see if it have same structure as the answer in that thread you following, if you does thing exactly right then it should solve your problem.

When I make predictions using this model, does it make predictions for 'Y' or a combination of Y and the additional feature?

The model will have same output shape exactly like what you defined, in your case because model output is Dense(1, activation="linear"), so it have output shape y_pred.shape == (batchsize, 1), nothing more, you can be sure about that, print it out using tf.print(y_pred) to see for yourself

also i don't know if it's your typing error, last line of your custom_loss function should be :

return K.mean(K.square((y_pred - y_true) + K.std(y_pred - feature)))

instead of

return K.mean(K.square((y_pred - y_true) + K.std(y__pred - feature)))
Mr. For Example
  • 4,173
  • 1
  • 9
  • 19
0

You can also use .add_loss with a simple mse loss the following way:

input = Input(size)
output = YourLayers(input)
model = Model(input, output)
model.add_loss(std(tf.gather(input, feature_idx, axis=1) - output))
model.compile(loss='mse', optimizer=opt)

BTW, it is strange that your regularizer is a square of variance, while your loss is mse. Maybe you would prefer them to be on the same squared scale (variance and mse), as people usually do (consider any L2 shrinkage, e.g. Ridge regression).

user2604507
  • 1