`gradient` gives AttributeError when building a TensorFlow graph

Question

I'm trying to learn how to build a graph in TensorFlow, but got stuck in a seemingly trivial operation. This is what I have,

import tensorflow as tf
def loss(x, y):
  tf.reduce_mean(tf.square(x - y))
xx = tf.random_normal([])
noise = tf.random_normal([])
yy = 3 * xx + 2 + noise
W = tf.get_variable("W", [])
W.assign(5)
b = tf.get_variable("b", [])
b.assign(0)
with tf.GradientTape() as t:
  current_loss = loss(W*xx+b, yy)
dW = t.gradient(current_loss, W)

At this point I got an AttributeError, as follows

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-7-26d05a240ccc> in <module>()
      1 with tf.GradientTape() as t:
      2   current_loss = loss(W*xx+b, yy)
----> 3 dW = t.gradient(current_loss, W)

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/backprop.py in gradient(self, target, sources, output_gradients, unconnected_gradients)
    944         flat_sources,
    945         output_gradients=output_gradients,
--> 946         unconnected_gradients=unconnected_gradients)
    947 
    948     if not self._persistent:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/imperative_grad.py in imperative_grad(tape, target, sources, output_gradients, unconnected_gradients)
     70       sources,
     71       output_gradients,
---> 72       compat.as_str(unconnected_gradients.value))

AttributeError: 'NoneType' object has no attribute '_id'

What am I doing wrong and how do I get the gradient? Thanks in advance.

Walfits · Answer 1 · 2019-02-22T17:01:28.273

The code above needs to be modified slightly if you don't want to use eager execution. When you want to use TensorFlow not in eager execution mode, you need to have two principal parts:

Building the graph
Running the graph

So I re-wrote the example above following this logic:

import tensorflow as tf
import numpy as np

# Building the graph
xx = tf.constant(np.random.normal(loc=0.0, scale=1.0))
noise = tf.constant(np.random.normal(loc=0.0, scale=1.0))
yy = 3 * xx + 2 + noise

W = tf.Variable(5.0, name="W")
b = tf.Variable(0.0, name="b")

current_loss = tf.reduce_mean(tf.square(tf.scalar_mul(W,xx)+b - yy))
dW = tf.gradients(current_loss, W, name='dW')[0]

# Initialisation operation
init = tf.global_variables_initializer()

# Creating a session and running the graph
with tf.Session() as sess:
    sess.run(init)

    dW_np = sess.run(dW)

print(dW_np)

I know that works. But the purpose of the exercise is building a graph without eager execution, and build custom NN layers from it. I'm not sure what caused the above error and how to avoid it. — jxy, Feb 22 '19 at 16:02
Sorry I misunderstood your question! I will update my answer. — Walfits, Feb 22 '19 at 16:34

score 1 · Accepted Answer · answered Feb 22 '19 at 19:01

1

Your loss() function doesn't return anything. That's why you have AttributeError (because current_loss is None). You should add return statement.

Regarding your comment for the previous answer. GradientTape is used in eager execution so you should add tf.enable_eager_execution() at the beginning of your program. If you want to build in graph mode you should use tf.gradients() or the compute_gradients() method of subclasses of tf.train.Optimizer(e.g. tf.train.GradientDescentOptimizer).

answered Feb 22 '19 at 19:01

Vlad

8,225
5
33
45

Am I not supposed to use `GradientTape` outside of eager execution mode? I didn't find any document mentioning this. – jxy Feb 22 '19 at 22:38
If you'd look at the traceback you'll see that it is defined in eager module: `/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/backprop.py`. See also answer [here](https://stackoverflow.com/questions/53953099/what-is-the-purpose-of-the-tensorflow-gradient-tape/53995313). – Vlad Feb 23 '19 at 07:18

`gradient` gives AttributeError when building a TensorFlow graph

2 Answers2