1

I'm trying to obtain the gradients from a keras model. The backend function keras.backend.gradients creates a symbolic function which needs to be evaluated on some specific input. The following code does work for this problem but it makes use of the old tensorflow sessions and in particular of feed_dict. 

import numpy as np
import keras
from keras import backend as K
import tensorflow as tf

model = keras.Sequential()
model.add(keras.layers.Dense(16, activation='relu', input_shape = (49, )))
model.add(keras.layers.Dense(11, activation='softmax'))
model.compile(optimizer='rmsprop', loss='mse')

trainingExample = np.random.random((1, 49))


gradients = K.gradients(model.output, model.trainable_weights)

sess = tf.InteractiveSession()
sess.run(tf.initialize_all_variables())

evaluated_gradients = sess.run(gradients,\
                               feed_dict={model.input:trainingExample})
sess.close()

How can I rewrite this in tensorflow 2 style, i.e. without the sessions? There is an alternative method described here. However I don't understand why it should be necessary to give some explicit output to evaluate the gradients and how to make the solution work without these outputs.

J.Unger
  • 11
  • 1

1 Answers1

1

In tensorflow-2 you can get gradients very easily using gradient tf.GradientTape().

I am citing the official tutorial code here - 

@tf.function
def train_step(images, labels):
  with tf.GradientTape() as tape:
    predictions = model(images)
    loss = loss_object(labels, predictions)
  gradients = tape.gradient(loss, model.trainable_variables)
  optimizer.apply_gradients(zip(gradients, model.trainable_variables))

  train_loss(loss)
  train_accuracy(labels, predictions)

you can find the complete tutorial in tensorflow official website - https://www.tensorflow.org/tutorials/quickstart/advanced

Mukul
  • 860
  • 8
  • 19