ValueError: No gradients provided for any variable in Tensorflow

Question

I'm trying to create a dice_loss function in Tensorflow. I'm facing a trouble with tensorlfow. Executing the following code

import tensorflow as tf
import tensorlayer as tl


def conv3d(x, inChans, outChans, kernel_size, stride, padding):
    weights = weight_variable([kernel_size, kernel_size, kernel_size, inChans, outChans])
    biases = bias_variable([outChans])
    conv = tf.nn.conv3d(x, weights, strides=[1, stride, stride, stride, 1], padding=padding)
    return tf.nn.bias_add(conv, biases)

def train(loss_val, var_list):
    optimizer = tf.train.AdamOptimizer(FLAGS.learning_rate)
    grads = optimizer.compute_gradients(loss_val, var_list=var_list)
    return optimizer.apply_gradients(grads)


def main(argv=None):
    image = tf.placeholder(tf.float32, shape=[None, SLICE_SIZE, IMAGE_SIZE, IMAGE_SIZE, 1], name="input_image")
    annotation = tf.placeholder(tf.float32, shape=[None, SLICE_SIZE, IMAGE_SIZE, IMAGE_SIZE, 1], name="annotation")

    logits, pred_annotation = vnet.VNet(image)
    loss = 1 - tl.cost.dice_coe(output=pred_annotation, target=annotation, axis=[1,2,3,4])

    trainable_var = tf.trainable_variables()
    train_op = train(loss, trainable_var)

    sess = tf.Session()
    ...

    ...

def VNet(x):
    ...
    out = tf.nn.elu(BatchNorm3d(conv3d(x, inChans, 2, kernel_size=5, stride=1, padding="SAME")))
    out = conv3d(out, 2, 2, kernel_size=1, stride=1, padding="SAME")
    annotation_pred = tf.to_float(tf.argmax(out, dimension=4, name='prediction'))
    return out, tf.expand_dims(annotation_pred, dim=4)

I get the following error:

ValueError: No gradients provided for any variable: ...

Someone can help me?

There's not enought code to work it out... How do you call your grads operation, what is in vnet, etc — Arnaud De Broissia, Jul 25 '17 at 08:21
Thank you!@Arnaud De Broissia, I modified the code, do you have some ideas? — Tramac, Jul 25 '17 at 08:56

score 4 · Answer 1 · answered Jul 25 '17 at 09:12

When you do annotation_pred = tf.to_float(tf.argmax(out, dimension=4, name='prediction')), you get an index of the max value in your tensor. This index can't be derivated, thus the gradient can't flow throught this operation.

So as your loss is only defined by this value, and the gradient can't flow throught it, no gradient can be calculated for your network.

I don't know specificately how the dice loss work, but maybe you wanted to use tf.max instead of tf.argmax, or you have to find a way to use an operation that can let the gradient flow.

I think your analysis is very reasonable！I will try your advice. Thank you very much! — Tramac, Jul 25 '17 at 09:21

ValueError: No gradients provided for any variable in Tensorflow

1 Answers1

Linked