2

I am trying to construct a custom loss for a regression problem with the following structure, following this answer: Keras Custom loss function to pass arguments other than y_true and y_pred

Now, my function is like the following:

def CustomLoss(model,X_valid,y_valid,batch_size):
    def Loss(y_true,y_pred):
        n_samples=5
        mc_predictions = np.zeros((n_samples,256,256))
        for i in range(n_samples):
           y_p = model.predict(X_valid, verbose=1,batch_size=batch_size)
    (Other operations...) 
        return LossValue
    return Loss

When trying to execute this line y_p = model.predict(X_valid, verbose=1,batch_size=batch_size) i get the following error:

Method requires being in cross-replica context, use get_replica_context().merge_call()

From what I gathered I cannot use model.predict inside loss function. Is there a workaround or solution for this? Please let me know if my question is clear or if you need any additional information. Thanks!

Eduardo
  • 67
  • 1
  • 11
  • I think you probably could hack it if you really wanted to. Essentially, the main issue is that predict is designed to call the static graph, and as such can't be a component of the static graph. I don't think you want/need to use predict as you likely already have all the data information in your X/Y to begin with, could you show why not or give some more context? – Andrew Holmgren Aug 12 '20 at 16:22
  • I would like to predict n_samples inside the loss function to generate a Bayesian approximation. So instead of comparing `y_true` and `y_pred` I can compare `y_true` vs `y_bayesian`. I hope this helps – Eduardo Aug 12 '20 at 16:26
  • Where y_bayesian is computed without gradients? Can you not just pass in y_bayesian to the custom loss? – Andrew Holmgren Aug 12 '20 at 21:03
  • y_bayesian changes at each epoch. That is why my first thought was to predict in the loss function to update y_bayesian. Is there any alternative to train in steps? Thanks – Eduardo Aug 13 '20 at 01:18
  • Yeah, you have a few options, you could do a custom training loop using GradientTape, or you could do a custom fit, or maybe you could even just put a conditional on the batch number that would update y_bayesian at the end of your epoch. – Andrew Holmgren Aug 14 '20 at 16:18
  • I will try the conditional at the end of the epoch to update. If I get a solution I will update here. Thanks – Eduardo Aug 17 '20 at 14:45

1 Answers1

1

Sounds like you can use model.add_loss for this. You can use this to specify the loss function inside of the model. It also removes the need for the loss function to only take in y and y_pred. https://www.tensorflow.org/api_docs/python/tf/keras/layers/Layer#add_loss \ Some psuedo-code:

class YourModel(tf.keras.Model): 
    ...
    def call(self, inputs): 
        unpack, any, extra, stuff = inputs
        (your network code goes here)
        loss = (other operations)
        self.add_loss(loss)
        return output

(In case you don't know, model.predict is basically just model.call but with some extra bells and whistles attached.)

Taw
  • 497
  • 3
  • 14
  • Thanks for the suggestion. I will definitively try. What is the advantage of using a class for the model? – Eduardo Aug 24 '20 at 02:05
  • Compared to using a sequential model, it's just a lot more customizable and flexible. Can also be nice to use an OOP approach. For simpler things it can end up being a little verbose. – Taw Aug 24 '20 at 04:52