2

I am trying to build a custom loss function in Keras, but I am confused about the way it works. I am training the network on batches, and I am not sure if the output of the loss function should be an array with the same dimension as the batch or just a scalar.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Denisa Ionascu
  • 33
  • 4
  • It should return the loss value per sample. The reduction of these values per batch is then taken care of by Keras. There is a detailed description mentioned in this post: https://stackoverflow.com/questions/63390725/should-the-custom-loss-function-in-keras-return-a-single-loss-value-for-the-batc/65128260#65128260 – Sanchit Dec 03 '20 at 14:55

2 Answers2

3

As described in the documentation, keras loss, You can pass a function that returns a scalar for each data-point and takes the two arguments: y_true (True labels) and y_pred (Predictions).

Keras execute the mean over sample inside batch, so the output should just be a single scalar.

Community
  • 1
  • 1
Deusy94
  • 733
  • 3
  • 13
1

The loss usually reduced over all dimensions of the mini-batch. If you don't apply reduction it would be performed implicitly (try removing tf.reduce_mean in custom_loss_function() and return just res). For example:

import tensorflow as tf
import numpy as np

def custom_cross_entropy(y_true, y_pred):
    res = -y_true*tf.math.log(tf.nn.softmax(y_pred))
    return tf.reduce_mean(res, axis=None)

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(2, activation=None))

model.compile(optimizer=tf.keras.optimizers.SGD(0.01),
              loss=[custom_cross_entropy],
              metrics=['accuracy'])

y_train = np.array([[1, 0], [0, 1]])
x_train = np.random.normal(size=(2, 2))

model.fit(x_train, y_train, epochs=2)

# Epoch 1/2
# 2/2 [==============================] - 0s 13ms/sample - loss: 0.2689 - accuracy: 1.0000
# Epoch 2/2
# 2/2 [==============================] - 0s 2ms/sample - loss: 0.2686 - accuracy: 1.0000
Vlad
  • 8,225
  • 5
  • 33
  • 45