Strange tensors

Question

What does it mean when I print a tensor and it shows some sort of operation name instead of its value?

Eg.:

Tensor("gradients/block1_conv1/convolution_grad/Conv2DBackpropInput:0", shape=(?, ?, ?, 3), dtype=float32)

Tensor("truediv_1:0", shape=(?, ?, ?, 3), dtype=float32)

The code that originates these prints is:

from keras.applications import VGG16
from keras import backend as K

model = VGG16(weights='imagenet',
              include_top=False)

layer_name = 'block3_conv1'
filter_index = 0

layer_output = model.get_layer(layer_name).output
loss = K.mean(layer_output[:, :, :, filter_index])

# The call to `gradients` returns a list of tensors (of size 1 in this case)
# hence we only keep the first element -- which is a tensor.
grads = K.gradients(loss, model.input)[0]


print("Print 1:")
print(grads)
print(type(grads))

# We add 1e-5 before dividing so as to avoid accidentally dividing by 0.
grads /= (K.sqrt(K.mean(K.square(grads))) + 1e-5)
grads /= (K.sqrt(K.mean(K.square(grads))) + 1e-5)
print("Print 2:")
print(grads)
print(type(grads))

According to the documentation a Tensor has no values, but means for reaching a final value in a given CPU or GPU session. At least as I understood it.

Is this what it really means?

How can I list all operations within a tensor, like in a sequential way, that take me from its input to the final value?

Eg. The grads tensor would be a gradient function, and two divisions, to calculate the final value.

Answer 1

Keras/Tensorflow (except when in eager mode, but I don't think Keras supports eager mode) is a "symbolic graph" language.

When you create a model and its operations, you are just creating a "graph", not the operations themselves. Then you only have the tensors.

The Keras models have "input tensors", which in Tensorflow are "placeholders". They are special tensors that "will" receive values when training or predicting. (In Keras: model.fit, model.predict, model.evaluate, etc. In Tensorflow, session.run)

Here an example of how to feed the inputs with data and get the actual values of the gradients: Getting gradient of model output w.r.t weights using Keras