Stack Overflow
Stack Overflow

Questions tagged [loss-function]

If Y_pred is very far off from Y, the Loss value will be very high. However, if both values are almost similar, the Loss value will be very low. Hence we need to keep a loss function which can penalize a model effectively while it is training on a dataset. When a neural network is trying to predict a discrete value, we can consider it to be a classification model. This could be a network trying to predict what kind of animal is present in an image, or whether an email is a spam or not.

1727 questions
128
votes
27 answers

NaN loss when training regression network

I have a data matrix in "one-hot encoding" (all ones and zeros) with 260,000 rows and 35 columns. I am using Keras to train a simple neural network to predict a continuous variable. The code to make the network is the following: model =…
The_Anomaly
  • 2,385
  • 3
  • 18
  • 22
109
votes
8 answers

L1/L2 regularization in PyTorch

How do I add L1/L2 regularization in PyTorch without manually computing it?
Wasi Ahmad
  • 35,739
  • 32
  • 114
  • 161
79
votes
12 answers

NotImplementedError: Cannot convert a symbolic Tensor (2nd_target:0) to a numpy array

I try to pass 2 loss functions to a model as Keras allows that. loss: String (name of objective function) or objective function or Loss instance. See losses. If the model has multiple outputs, you can use a different loss on each output by passing…
T D Nguyen
  • 7,054
  • 4
  • 51
  • 71
77
votes
2 answers

How does keras handle multiple losses?

If I have something like: model = Model(inputs = input, outputs = [y1,y2]) l1 = 0.5 l2 = 0.3 model.compile(loss = [loss1,loss2], loss_weights = [l1,l2], ...) what does Keras do with the losses to obtain the final loss? Is it something…
jfga
  • 803
  • 1
  • 8
  • 8
46
votes
6 answers

RMSE/ RMSLE loss function in Keras

I try to participate in my first Kaggle competition where RMSLE is given as the required loss function. For I have found nothing how to implement this loss function I tried to settle for RMSE. I know this was part of Keras in the past, is there any…
dennis
  • 707
  • 1
  • 8
  • 12
40
votes
2 answers

What are the differences between all these cross-entropy losses in Keras and TensorFlow?

What are the differences between all these cross-entropy losses? Keras is talking about Binary cross-entropy Categorical cross-entropy Sparse categorical cross-entropy While TensorFlow has Softmax cross-entropy with logits Sparse softmax…
ScientiaEtVeritas
  • 5,158
  • 4
  • 41
  • 59
39
votes
4 answers

How do I mask a loss function in Keras with the TensorFlow backend?

I am trying to implement a sequence-to-sequence task using LSTM by Keras with the TensorFlow backend. The inputs are English sentences with variable lengths. To construct a dataset with 2-D shape [batch_number, max_sentence_length], I add EOF at the…
Shuaaai
  • 403
  • 1
  • 4
  • 8
28
votes
3 answers

What function defines accuracy in Keras when the loss is mean squared error (MSE)?

How is Accuracy defined when the loss function is mean square error? Is it mean absolute percentage error? The model I use has output activation linear and is compiled with loss= mean_squared_error model.add(Dense(1)) model.add(Activation('linear'))…
Endre Moen
  • 695
  • 2
  • 9
  • 19
27
votes
7 answers

Should the custom loss function in Keras return a single loss value for the batch or an arrary of losses for every sample in the training batch?

I'm learning keras API in tensorflow(2.3). In this guide on tensorflow website, I found an example of custom loss funciton: def custom_mean_squared_error(y_true, y_pred): return tf.math.reduce_mean(tf.square(y_true - y_pred)) The…
Gödel
  • 592
  • 4
  • 21
25
votes
2 answers

How does TensorFlow SparseCategoricalCrossentropy work?

I'm trying to understand this loss function in TensorFlow but I don't get it. It's SparseCategoricalCrossentropy. All other loss functions need outputs and labels of the same shape, this specific loss function doesn't. Source code: import tensorflow…
Dee
  • 7,455
  • 6
  • 36
  • 70
24
votes
1 answer

How to iterate through tensors in custom loss function?

I'm using keras with tensorflow backend. My goal is to query the batchsize of the current batch in a custom loss function. This is needed to compute values of the custom loss functions which depend on the index of particular observations. I like to…
Thomas
  • 350
  • 2
  • 8
20
votes
2 answers

What would be a good loss function to penalize the magnitude and sign difference

I'm in a situation where I need to train a model to predict a scalar value, and it's important to have the predicted value be in the same direction as the true value, while the squared error being minimum. What would be a good choice of loss…
Ethan Chen
  • 305
  • 2
  • 7
20
votes
1 answer

Custom weighted loss function in Keras for weighing each element

I'm trying to create a simple weighted loss function. Say, I have input dimensions 100 * 5, and output dimensions also 100 * 5. I also have a weight matrix of the same dimension. Something like the following: import numpy as np train_X =…
Nipun Batra
  • 11,007
  • 11
  • 52
  • 77
17
votes
1 answer

Custom loss function in Keras to penalize false negatives

I am working on a medical dataset where I am trying to have as less false negatives as possible. A prediction of "disease when actually no disease" is okay for me but a prediction "no disease when actually a disease" is not. That is, I am okay with…
Swapnil B.
  • 729
  • 1
  • 8
  • 23
15
votes
3 answers

Keras custom loss as a function of multiple outputs

I built a custom architecture with keras (a convnet). The network has 4 heads, each outputting a tensor of different size. I am trying to write a custom loss function as a function of this 4 outputs. I have been implementing cusutom losses before,…
defqoon
  • 173
  • 1
  • 2
  • 7
1
2 3
…
99 100