How to use the BCELoss in PyTorch?

Question

I want to write a simple autoencoder in PyTorch and use BCELoss, however, I get NaN out, since it expects the targets to be between 0 and 1. Could someone post a simple use case of BCELoss?

https://github.com/pytorch/examples/blob/master/vae/main.py#L80-L85 — fonfonx, May 01 '17 at 14:53
Have you added the sigmoid function for the last layer in your network? — Kashyap, May 03 '17 at 09:28

score 20 · Accepted Answer · edited Aug 04 '19 at 17:14

Update

The BCELoss function did not use to be numerically stable. See this issue https://github.com/pytorch/pytorch/issues/751. However, this issue has been resolved with Pull #1792, so that BCELoss is numerically stable now!

Old answer

If you build PyTorch from source, you can use the numerically stable function BCEWithLogitsLoss(contributed in https://github.com/pytorch/pytorch/pull/1792), which takes logits as input.

Otherwise, you can use the following function (contributed by yzgao in the above issue):

class StableBCELoss(nn.modules.Module):
       def __init__(self):
             super(StableBCELoss, self).__init__()
       def forward(self, input, target):
             neg_abs = - input.abs()
             loss = input.clamp(min=0) - input * target + (1 + neg_abs.exp()).log()
             return loss.mean()

score 5 · Answer 2 · answered May 05 '17 at 18:34

5

You might want to use a sigmoid layer at the end of the network. In that way the number would represent probabilities. Also make sure that the targets are binary numbers. If you post your complete code we might help more.

answered May 05 '17 at 18:34

Roger Trullo

1,436
2
10
19

How to use the BCELoss in PyTorch?

2 Answers2

Update

Old answer