1

According to Keras Code, Keras computes the loss value considering optional weights and masks.

       for i in range(len(self.outputs)):
        if i in skip_target_indices:
            continue
        y_true = self.targets[i]
        y_pred = self.outputs[i]
        weighted_loss = weighted_losses[i]
        sample_weight = sample_weights[i]
        mask = masks[i]
        loss_weight = loss_weights_list[i]
        with K.name_scope(self.output_names[i] + '_loss'):
            output_loss = weighted_loss(y_true, y_pred,
                                        sample_weight, mask)
        if len(self.outputs) > 1:
            self.metrics_tensors.append(output_loss)
            self.metrics_names.append(self.output_names[i] + '_loss')
        if total_loss is None:
            total_loss = loss_weight * output_loss
        else:
            total_loss += loss_weight * output_loss

On the other hand, in Keras documentation I see the basic loss function is introduced in compile function, and then sample or class weights can be introduced in fit command. I am not sure how to relate 'weights and masks' in the first code to 'sample and class weights' in the second document. Can anybody give me more explanation?

My application is actually a Convolutional LSTM network, where I input a series of images and want the network to produce an output map (with the same size of input maps) of pixel classes, but some pixels don't have valid labels during training. Should I use weight or mask, sample or class?

Shahriar49
  • 621
  • 1
  • 5
  • 18
  • You should use masking if some pixels don't have a valid label, so that to ignore them in computation of loss (however, I am not sure Keras supports masking for convolution layers). See [this answer](https://stackoverflow.com/a/52570297/2099607) and [this answer](https://stackoverflow.com/a/53470422/2099607) to get a general idea of how masking works in Keras. – today Aug 15 '19 at 10:57
  • I looked at the links but my understanding is that masking has nothing to do with invalid labels directly. Masking just masks some elements of input samples from affecting the output by disabling them from the entering the network (or at any layer the masking is introduced). So this is a within-sample issue. But the sample weighting is a sample-level issue and class weight is a class-level issue, which are three different levels of data. – Shahriar49 Aug 15 '19 at 12:42
  • My understanding is that if the labels are "invalid" as you suggest, then the samples associated with them should not affect the learning and therefore be masked. However, instead you can also play with sample weights and/or class weights if you have an idea to achieve the same effect. And of course, I know very well that sample weights, class weights and masking are three distinct concepts used for different purposes. What I am not sure about is what you mean by "invalid" labels here, and how you want to change/affect learning process. – today Aug 15 '19 at 13:30
  • By 'invalid' labels I mean that some pixels have not a valid label associated with them, but the other pixels in the image have (it is a pixel classification application). The problem with masking in convolution is that no pixel itself is invalid to be masked, as each pixel is used to do convolution calculation of nearby pixels. Just the final labels (ground truth) is not complete and has holes in it. So the calculations should be done as usual, just the loss function needs to be aware of the holes in labels. – Shahriar49 Aug 15 '19 at 14:56
  • Ah, I see now. Keras does not support masking or weighting in that level. One solution could be to provide masks as another input of the model, and then define a **custom** loss function which incorporate the given masks in the computation of loss. – today Aug 15 '19 at 15:28
  • At least in theory it could be possible, by flattening each input sample and using 'sample_weight_mode' to temporal, and then supply the desired masking pattern as sample weights along with input dataset. But I need more confidence for correctness of this approach. – Shahriar49 Aug 15 '19 at 15:37
  • Also I am still confused about my first question mentioned in my post: I see no masking input in compile or fit commands, just see sample_weight and class_weight. In Keras code I showed above, the loss is influenced by mask, sample_weight and loss_weight. There is no mask in the first case and class_weight in second case. – Shahriar49 Aug 15 '19 at 15:41
  • Oh, right; that's also one solution, probably much easier. I forgot that there is a temporal model for sample weights. And if it's theoretically correct (at least it seems to be), then you should experiment to find out if it helps with your specific problem/dataset. – today Aug 15 '19 at 15:41
  • Masks are not given as input to models in Keras; instead it will be set using `Masking` layer or the mask arguments of some of the layers (like embedding layer). – today Aug 15 '19 at 15:42
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/197995/discussion-between-shahriar49-and-today). – Shahriar49 Aug 15 '19 at 15:55

0 Answers0