0

I've essentially got a keras CNN with a few layers and I want to evaluate one input then get the gradients of the backpropagation algorithm. The model is set up and compiled. But I only want the first time backpropagation is run on just one input/output set, I don't care about anything else.

Here's the relevant part of my code:

# Build the model.
model = Sequential()
model.add(Conv2D(16, kernel_size=(3, 3), activation='relu', input_shape=input_shape))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(digits, activation='softmax'))

# Prepare the model for training.
model.compile(loss = keras.losses.categorical_crossentropy,
              optimizer = keras.optimizers.Adadelta(),
              metrics=['accuracy'])

And my one set of data input and output is in x_train and y_train. So how exactly do I run that one set of data and run backpropagation based on the output expected, then actually get the gradients that keras calculated?

EDIT: Essentially, this is an academic endeavor and what I'm trying to do is calculate the values of the gradients myself, then compare those to what keras gets. So how do I get those values?

figure09
  • 19
  • 1
  • 4

1 Answers1

0

You should be able to use the gradients function from the Keras backend. 

# Gradients of output wrt input
gradients = K.gradients(model.output, model.input)

# Wrap the input tensor and the gradient tensor in a callable function
f = K.function([model.input], gradients)

# Random input image
x = np.random.rand(1, 100,100,3)

f([x])
> gives an array of shape (1, 100, 100, 3)

Edit: to get the gradients of the output wrt the model weights, you can use the following code:

# List all the weight tensors in the model
weights_list = model.trainable_weights

# Gradients of output wrt model weights
gradients = K.gradients(model.output, weights_list)

# Wrap the model input tensor and the gradient tensors in a callable function
f = K.function([model.input], gradients)

# Random input image
x = np.random.rand(1,100,100,3)

# Call the function to get the gradients of the model output produced by this image, wrt the model weights
f([x])

Edit: Your question probably has an answer here.

sdcbr
  • 7,021
  • 3
  • 27
  • 44
  • I'm sure I can, I just probably don't quite understand how it works. From a theory stand point, I should get one gradient value per weight I have. So that's what I'm looking for.. Or maybe even just one layer would be enough. – figure09 Feb 01 '19 at 18:47
  • So I definitely got these values, in two different ways, I'm just not quite sure what they represent. Am I wrong in my earlier comment to expect 1 value per edge weight? What I'm getting now is definitely not that. – figure09 Feb 01 '19 at 18:55
  • Updated my answer. – sdcbr Feb 02 '19 at 19:05