How to print the value of a tensor in tensorflow mnist_softmax.py

Question

I just tried to run mnist_softmax.py in TensorFlow 0.8. I want to observe the value of y and y_ just before the model test step.

Below is the code:

print(y)  # added by me
print(y_)  # added by me

# Test trained model
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))

The completed code is available on GitHub.

Below is the output:

Tensor("Softmax:0", shape=(?, 10), dtype=float32)
Tensor("Placeholder_1:0", shape=(?, 10), dtype=float32)

I have also tried to use sess.run(y), and y.eval(), but I get an error like this when I try it:

tensorflow.python.framework.errors.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype float
         [[Node: Placeholder = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
Caused by op u'Placeholder', defined at: ...

Answer 1

TL;DR: Both the y and y_ tensors depend on tf.placeholder() operations, and so they require you to feed an input value when you evaluated them. You can print the output of the softmax on a batch of input data by feeding a batch of input like so:

batch_xs, _ = mnist.train.next_batch(100)
print(y.eval({x: batch_xs}))

---

The MNIST example contains the following lines:

# Create the model
x = tf.placeholder(tf.float32, [None, 784])
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
y = tf.nn.softmax(tf.matmul(x, W) + b)

Note that the first tensor you are trying to print, y, is a function of x, which is defined as a tf.placeholder(). The tf.placeholder() op is a way of defining a symbolic argument to the computation: it doesn't have any value itself, but instead it says that x must be a matrix of floating-point values with 784 columns.

Running/evaluating y without providing a value for x is like writing the following Python function and calling it without all of its arguments:

def y(x):
    W = ...
    b = ...
    return softmax(matmul(x, W), b)

# This would fail with an error.
print(y())

How do you specify a value for the argument? In TensorFlow, you do this by feeding a value for the placeholder (like in the training loop and accuracy calculation):

# Get a batch of input data to feed to the computation.
batch_xs, _ = mnist.train.next_batch(100)

print(sess.run(y, feed_dict={x: batch_xs}))
# or
print(y.eval({x: batch_xs}))

The tensor y_ is defined as a tf.placeholder(). For technical reasons, you can't evaluate a placeholder directly, even if you feed a value for it. However, it wouldn't be particularly useful to do this! Instead, you can just print the value that you would have fed.