How do I print inside the loss function during training in Keras?

Question

I am trying to create a loss function in Keras (Tensorflow Backend) but I am a little stuck to check the inside of the custom loss function. In fact, the print appears on the console only when I compile the model, after that there is no print. (I am just testing very simple custom function, I will create the true function when I solved this problem). I train the model using the train_on_batch function. How can I solve this problem?

def loss_yolo(self, y_true, y_pred):  
    print('inside loss function')
    loss = K.mean(y_true - y_pred)
    return loss

model.compile(optimizer='sgd', loss=loss_yolo)

print('train on batch')
print(model.train_on_batch(x, y))

And the output is

inside loss function

train on batch

-0.481604

I tried with the function x = tf.Print( ...) and then print(x.eval()) but I have the same problem — Shiro, Apr 13 '18 at 15:49
This will not work. And there is not much you can do. These functions are used to create a "graph", and all calculations are made internally by or in the graph. Prints are not part of the graph. — Daniel Möller, Apr 13 '18 at 17:07

score 10 · Answer 1 · answered Apr 13 '18 at 18:58

10

The only thing you can do is not use python's print function, but for example, tensorflow's tf.Print function that is part of the computational graph. The documentation says the operation does nothing but each time it is evaluated it prints a message that you can specify.

You just need to be careful to place it correctly in the graph, something like:

def loss(y_true, y_pred):
    d = y_true - y_pred
    d = tf.Print(d, [d], "Inside loss function")
    return tf.reduce_mean(tf.square(d))

A better option to look inside what is going on internally is to use the tensorflow debugger.

answered Apr 13 '18 at 18:58

Dr. Snoopy

55,122
7
121
140

How to print full tensor? in my case it print just `[[1 1 1]...]` – mrgloom Nov 19 '18 at 11:48
@mrgloom It's saying `[1 1 1]` *is* the value of your tensor - are you expecting something else? – flow2k Aug 23 '19 at 07:53
1

With default params it doesn't print all tensor values with `summarize=1000` it does. – mrgloom Aug 23 '19 at 09:16
6

This no longer works, even replacing `tf.Print` with `tf.print`. What's the new solution? – Rylan Schaeffer Mar 08 '20 at 00:44
4

@RylanSchaeffer, right. With tensorflow 2.0, tf.print() seems to not print anything – MasterControlProgram May 05 '20 at 12:56

score 4 · Answer 2 · answered Jan 31 '21 at 17:29

I added the output_stream argument and tried this code in TensorFlow v2.4.1. Worked fine:

def loss_custom(y_true, y_pred):
    d = y_true - y_pred
    tf.print("\n y_true:", type(y_true), output_stream=sys.stdout)
    return tf.reduce_mean(tf.square(d))

Output during training:

Epoch 1/10

 y_true: <class 'tensorflow.python.framework.ops.Tensor'>
 1/72 [..............................] - ETA: 0s - loss: 0.2328 - accuracy: 0.3319
 y_true: <class 'tensorflow.python.framework.ops.Tensor'>
 2/72 [..............................] - ETA: 9s - loss: 0.2087 - accuracy: 0.5250
 y_true: <class 'tensorflow.python.framework.ops.Tensor'>

How do I print inside the loss function during training in Keras?

2 Answers2

Linked