Tensorflow: why doesn't tensorflow print each value in this loop?

Question

I was going through some Tensorflow code that runs a tf.while_loop() and I had a question. The code calculates the roots of a 4th degree polynomial as part of a lab for learning tensorflow. My question is why a specific print statement will not output all of the intermediate values.

The code is below:

import tensorflow as tf

def f(x, w):
    return (w[0] + w[1] * x + w[2] * tf.pow(x,2) + w[3] * tf.pow(x,3) + w[4] * tf.pow(x,4) )

def f1(x, w):
    return (w[1] + 2. * w[2] * x + 3. * w[3] * tf.pow(x,2) + 4. * w[4] * tf.pow(x,3) )

def f2(x, w):
    return (2. * w[2] + 6. * w[3] * x + 12. * w[4] * tf.pow(x,2) )

def fxn_plus_1(xn, w):
    return (xn - (2. * f(xn, w) * f1(xn, w) / (2. * tf.square(f1(xn, w)) - f(xn, w) * f2(xn, w))))

def c(x, weights):
    return tf.abs(x - fxn_plus_1(x, weights)) > 0.001

def b(x, weights):
    x = fxn_plus_1(x, weights)
    return x, weights

weights = tf.constant( [-1000., 1., 1. , 5. , 0.1])

x = fxn_plus_1(-10., weights)

out = tf.while_loop(c, b, [x, weights])

with tf.Session() as sess:
    x, weights = sess.run(out)
    print(x)

The output is correct with a values of 5.575055. Now, I wanted to see what the intermediate values were of the loop body b() as the algorithm proceeded. I changed the function b() to the following:

def b(x, weights):
    x = fxn_plus_1(x, weights)
    print(x) ## ADDED PRINT STATEMENT
    print(weights)
    return x, weights

What I get in return is:

Tensor("while_4/sub_4:0", shape=(), dtype=float32)
Tensor("while_4/Identity_1:0", shape=(5,), dtype=float32)
5.575055

this seems to give the debug output or the graph information for the values of x,weights instead of the actual values. I was not sure how to get the loop to actually print the values at each step.

Any suggestions? Thanks.

UPDATE on things attempted and suggested:

One of the commentors @user49593, suggested I try using tf.Print().

Here is the code, and the output. I am still just getting the graph info instead of the actual vector of values.

def b(x, weights):
    x = fxn_plus_1(x, weights)
    x = tf.Print(x, [x], message="here: ") #CHANGED TO tf.Print STATEMENT
    return x, weights

The output is still just 5.575055. No vector of intermediate values.

Possible duplicate of [How to print the value of a Tensor object in TensorFlow?](https://stackoverflow.com/questions/33633370/how-to-print-the-value-of-a-tensor-object-in-tensorflow) — IanQ, Oct 23 '18 at 19:21
@user49593 Thanks for checking, but I think this is a different issue. I read the link, and it suggests using the `eval()` method. But that is used if you are evaluating a function inside of a `with tf.Session() as sess:`. In the OP, the point is that the print statement in the `b()` function is setup before the session context, but that print statement is called from within the session context. I tried using `eval()` and got an error. — krishnab, Oct 23 '18 at 19:30
Did you try using `tf.Print()` as mentioned? "To print the value of a tensor without returning it to your Python program, you can use the tf.Print() operator" — IanQ, Oct 23 '18 at 19:38
I did try `tf.Print()` but it is still just giving me the graph output instead of the actual values. I can update the OP with the output I get when using `tf.Print()`. Yeah, it is just confusing why this issue seems so confusing. I am used to the tricks that you mentioned when trying to debug in the `sess.run()` but these are functions called within `sess.run()`. — krishnab, Oct 23 '18 at 19:50
That's not how you use `tf.Print()`. Do `x = tf.Print(x, [x], message="here: ")` [Using tf.Print blog post](https://towardsdatascience.com/using-tf-print-in-tensorflow-aa26e1cff11e) — IanQ, Oct 23 '18 at 19:58
Sorry about that. I changed the code to exactly as you suggested, but still no change in the output. I will update the OP. — krishnab, Oct 23 '18 at 20:00
I am actually doing this oon google Colab. I have an jupyter notebook open there. I can share it with you? Would that make sense. — krishnab, Oct 23 '18 at 20:21
If I remember correctly, you can't do `tf.Print()` in jupyter - it just doesn't output anything — IanQ, Oct 23 '18 at 20:23
Interesting, I had not heard that. You think I should try and run it in Tensorboard and see if I can see intermediate results there? I found this on the jupyter issue: https://stackoverflow.com/questions/37898478/is-there-a-way-to-get-tensorflow-tf-print-output-to-appear-in-jupyter-notebook-o — krishnab, Oct 23 '18 at 20:25
@user49593 Hey, thanks for the tip about notebooks. Turns out that the code works when I run the the code as a file, but not when I run it in a notebook. But glad that it was not something more exotic. — krishnab, Oct 24 '18 at 01:48

score 0 · Answer 1 · answered Oct 24 '18 at 21:04

The answer to this question, was really found by @user49593. The problem is that the Jupyter Notebooks don't show the output of the print statements. So in order to see the output, you need to put the code into a file and then run it from the command line.

So if I wrap up the code above into a file named equation.py. Then I would go to the command line and type.

python equation.py

This will generate the expected output. Looks like there is another SE post related to this, which might have some additional insights.

Is there a way to get tensorflow tf.Print output to appear in Jupyter Notebook output

Tensorflow: why doesn't tensorflow print each value in this loop?

1 Answers1