2

I'm currently using the Cross Entropy Loss function but with the imbalance data-set the performance is not great.

Is there better lost function?

Shai
  • 111,146
  • 38
  • 238
  • 371
Fractale
  • 1,503
  • 3
  • 19
  • 34

1 Answers1

11

It's a very broad subject, but IMHO, you should try focal loss: It was introduced by Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He and Piotr Dollar to handle imbalance prediction in object detection. Since introduced it was also used in the context of segmentation.
The idea of the focal loss is to reduce both loss and gradient for correct (or almost correct) prediction while emphasizing the gradient of errors.

As you can see in the graph:
enter image description here

Blue curve is the regular cross entropy loss: it has on the one hand non-negligible loss and gradient even for well classified examples, and on the other hand it has weaker gradient for the erroneously classified examples.
In contrast, focal loss (all other curves) has smaller loss and weaker gradient for the well classified examples and stronger gradients for the erroneously classified examples.

Shai
  • 111,146
  • 38
  • 238
  • 371
  • 1
    This seems to be a good approach for a binary class problem. However the question is about a multi-class problem. Is there any extension of the same idea for the multi-class problem? – Tides Jan 13 '20 at 12:03
  • 1
    @Tides although the paper states the loss in the context of *binary* cross entropy, it is quite trivial to extend it to multi-label cross entropy. – Shai Jan 13 '20 at 14:20