I don't completely understand how to return the calculated loss in a custom loss function. As I know the returned value should be scalar. So when calculating loss in the function should I consider that there is a batch with multiple samples fed to this function? or I should write the code for calculating loss for just one sample and Tensorflow will take the mean of the returned value if finds out there are multiple inputs on the loss function returned value? I found two codes, one Huber Loss and another Contrastive Loss. they both works. But Huber loss function doesn't take mean of the output (dosen't consider there are multiple sample input and for a fiven batch input will return a vector) and the Contrastive loss considered it and took a mean to return scalar. Apparently both of them work.
Here is one which returns a vector for a batch input:
# inputs
y_true = tf.constant([-1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
y_pred = tf.constant([-3.0, -1.0, 1.0, 3.0, 5.0, 7.0], dtype=float)
def huber_loss(y_true, y_pred):
threshold = 1
error = y_true - y_pred
is_small_error = tf.abs(error) <= threshold
small_error_loss = tf.square(error) / 2
big_error_loss = threshold * (tf.abs(error) - (0.5 * threshold))
return tf.where(is_small_error, small_error_loss, big_error_loss)
And another which returns a scalar (mean of the values for a batch input):
def contrastive_loss(y_true, y_pred):
margin = 1
square_pred = K.square(y_pred)
margin_square = K.square(K.maximum(margin - y_pred, 0))
return K.mean(y_true * square_pred + (1 - y_true) * margin_square)
So should I take mean of the output to get the scalar value from all the batch samples or Tensorflow do it?
