4

Is it possible to intercept a backwards gradient from some arbitrary layer, modify its values and continue the backpropagation back to the start of network, updating the back gradients of all previous layers based off the modified gradient values you provided?

I know that you can directly modify the gradients themselves before applying the update, but as far as I know, that would only modify the gradients of the specified layer without propagating to previous layers' gradients.

Yan Zuo
  • 41
  • 1

1 Answers1

1

You should be able to create a modified custom gradient operation that does this for you, using the approach that's described here: Tensorflow: How to replace or modify gradient?

Pete Warden
  • 2,866
  • 1
  • 13
  • 12