tf.print() vs Python print vs tensor.eval()

Question

It seems that in Tensorflow, there are at least three methods to print out the value of a tensor. I've been reading here and there, yet still very confused.

These authors seem to summarize the different usage as:

• Python print: can only print out certain attributes of a tensor, e.g. its shape, because outside the computation graph it's merely an operation.
• tensor.eval() : Not sure its difference with tf.print()
• tf.print(): can output the actual value of a tensor, but must be inserted somewhere in the graph as well as being used by some other operation otherwise it will be dangling and still not printed.

My confusion probably also lies in I'm not sure how much Python functionalities we can access in a tensorflow computation graph, or where the computation graph "ends" and Python code "begins". e.g.

1. If I insert a Python print between two lines where I construct a computation graph, when I call sess.run() later, will this line be called?
2. If I want to print out multiple tensor values in a computation graph, where should I put these statements?
3. What's the difference between tensor.eval() and tf.print()? How should I use them differently?

Answer 1

The native Python print() statement will be called when the graph is build the first time. Check this out:

a = tf.placeholder(shape=None, dtype=tf.int32)
b = tf.placeholder(shape=None, dtype=tf.int32)
print("a is ",a," while b is ",b)
c = tf.add(a, b)
with tf.Session() as sess:
    print(sess.run(c, feed_dict={a: 1, b: 2}))
    print(sess.run(c, feed_dict={a: 3, b: 1}))

By exectuing this code block, the output is:

# a is  Tensor("Placeholder:0", dtype=int32)  while b is  Tensor("Placeholder_1:0", dtype=int32)
# 3
# 4

On the other hand, let us see tf.print():

a = tf.placeholder(shape=None, dtype=tf.int32)
b = tf.placeholder(shape=None, dtype=tf.int32)
print_op = tf.print("a is ",a," while b is ",b)
with tf.control_dependencies([print_op]):
    c = tf.add(a, b)

with tf.Session() as sess:
    print(sess.run(c, feed_dict={a: 1, b: 2}))
    print(sess.run(c, feed_dict={a: 3, b: 1}))

So, according to the output below, we can see that if we add the dependency that the tf.print op must be run whenever c is run, we get to see the output we want to:

# a is  1  while b is  2
# 3
# a is  3  while b is  1
# 4

Finally, tensor.eval() is identical to sess.run(tensor). However, the limitation of tensor.eval() is that you can run it to evaluate a single tensor, while the tf.Session can be used to evaluate multiple tensors sess.run([tensor1, tensor2]). If you ask me, I would always use sess.run(list_of_tensors), to evaluate as many tensors as I want to, and print out their values.

Answer 2

1. No. The Python print is not called when you call sess.run() later.
   If you want to print when you call sess.run() then you can use tf.print.

2. To print out multiple tensor values in a graph, you should use sess.run() after opening tf.Session(). Sample code is below.

t = tf.constant(42.0)
u = tf.constant(37.0)
pt = tf.print(t)
pu = tf.print(u)
with sess.as_default():
   sess.run([pt, pu])

42
37

3. This answer and this in another question will be helpful.
   tensor.eval() evaluates tensor operation and is not an operator.
   tf.print() is just an operator that prints out given tensor. So after invoking tf.Session(), tf.print() is to be one of the graph nodes.