2

I am using TensorFlow and Python to create custom neural network. I need to make changes on weights and biases before each input data is passed. The architecture of network is common (sequential, supervised, with backpropagation) and the only difference is that before each pass I need to make some calculations.

So for example I have some inputs (x) and before I pass them throught the network and calculate network results (y), in each pass I need to run a function to change the weights. My question is how would be possible to do that and after I calculate new weights and biases, that the network calculates everything else (whole network calculation, loss, and optimization functions) further normally? If it is possible how could I reach weights and then create that additional custom step?

double-beep
  • 5,031
  • 17
  • 33
  • 41
Ivan Gradecak
  • 23
  • 4

2 Answers2

1

You can use tf.assign operation. The link is here. This blog by metaflow can also be useful.

Abhishek Mishra
  • 1,984
  • 1
  • 18
  • 13
1

You should first gather all weights which should be changed in a collection like this way or select the variables by their names, and then you can do the change and then assign the changed values to the original variables. After the sess.run you can change them in reality.

For instance: 

t_vars = tf.trainable_variables()
d_vars = [var for var in t_vars if 'this_' in var.name]
clip_D = [p.assign(tf.clip_by_value(p, -1, 1)) for p in d_vars]
Lerner Zhang
  • 6,184
  • 2
  • 49
  • 66