I'm working with a simple MLP which uses MSE as a loss function for its scalar target "y_true". However, MSE does not really capture the error of "y_pred" compared to "y_true".
In this case, a more suitable loss function would be based on another variable, lets call it "y_eval", as follows
loss_1(y_true,y_pred,y_eval) = max(0,y_true - y_eval) if y_pred <= y_eval
else max(0,y_eval - y_true),
or even squared
loss_2(y_true,y_pred,y_eval) = max(0,y_true - y_eval)^2 if y_pred <= y_eval
else max(0,y_eval - y_true)^2.
What would be the best way to implement it? And is this a valid loss function? I'm thinking about creating a new target [y_true, y_eval] with shape (2,) and going from there... Thanks for any advice!