Unique identifiers for TensorFlow 2 tensors?

Question

I have a need to be able to tell tensors clearly apart from each other in my debugging output. Let me illustrate this with an example problem:

import tensorflow as tf

def loss(x):
    return x**2

x = tf.Variable(5,dtype=float)
y = tf.Variable(x,dtype=float)

print("x:", x)
print("y:", y)

with tf.GradientTape() as tape1:
    z1 = loss(x)
grad_z1 = tape1.gradient(z1, [x])

with tf.GradientTape() as tape2:
    z2 = loss(y)
grad_z2 = tape2.gradient(z2, [x]) # Variable should be y here!

print("grad_z1:", grad_z1)
print("grad_z2:", grad_z2)

where the output is:

x: <tf.Variable 'Variable:0' shape=() dtype=float32, numpy=5.0>
y: <tf.Variable 'Variable:0' shape=() dtype=float32, numpy=5.0>
grad_z1: [<tf.Tensor: id=25, shape=(), dtype=float32, numpy=10.0>]
grad_z2: [None]

Here, I am trying to get the gradient of a simple loss function with respect to some input variable. In the "z1" case, the example works fine because there is a graph connection from x to z1. However it breaks in the z2 case, because there is no graph connection from x to z2. This connection was "accidentally" broken by initialising a new variable, y, from the value of x. The problem is obvious in this example, but in my much more complicated real code it is a lot easier to accidentally replace variables like this, destroying the calculation. I then have to dig around trying to figure out where I made some mistake like this.

This process would be a lot easier if I could inspect the tensors and find out where they become different objects. For example, is there some sort of unique ID property or something that I can inspect? In the example above, I cannot tell that x and y are in fact completely different variables from the printed output. They look identical, but of course are not.

So I need something else that I can print to help track down when x accidentally was swapped for y. Is there any such property? Surely there must be, but I cannot find one. Maybe I could print the memory address of the objects or something instead, but I'm not sure how to do that in Python either.

score 0 · Answer 1 · answered Jun 04 '20 at 02:36

Well, it seems that Python has an in-built mechanism like I want: (Accessing Object Memory Address):

id(object)

This is probably good enough to solve my problem. Though if there is a simpler identifier to look at then I guess I would prefer it to staring at these random-looking large numbers.

score 0 · Answer 2 · answered Jun 04 '20 at 10:54

I don't know if it helps, but you can name your tensors and variables in TensorFlow, see in the docs. So in your case, you would do something like:

x = tf.Variable(5,dtype=float, name='x')
y = tf.Variable(x,dtype=float, name='y')

If this is still not enough, for example if your tensors would have the same names, you can use this:

import tensorflow.keras.backend as K

x = tf.Variable(5,dtype=float, name='x_' + str(K.get_uid('x')))
y = tf.Variable(x,dtype=float, name='x_' + str(K.get_uid('x')))

Unique identifiers for TensorFlow 2 tensors?

2 Answers2