How to print weights in Tensorflow?

Question

I am trying to get the weight matrix of my hidden_layer2 and print it. It seems like I am able to get the weight matrix, but I am not able to print it.

When using tf.Print(w, [w]) it prints nothing. When using print(tf.Print(w,[w]) it prints at least the info about the tensor:

Tensor("hidden_layer2_2/Print:0", shape=(3, 2), dtype=float32)

I also tried to use tf.Print() outside of the with-Statement, same result.

Full code is here, I am just processing random data in a feed-forward NN: https://pastebin.com/KiQUBqK4

A part of my Code:

hidden_layer2 = tf.layers.dense(
        inputs=hidden_layer1,
        units=2,
        activation=tf.nn.relu,
        name="hidden_layer2")


with tf.variable_scope("hidden_layer2", reuse=True):
        w = tf.get_variable("kernel")
        tf.Print(w, [w])
        # Also tried tf.Print(hidden_layer2, [w])

Did you run the graph afterwards? `tf.Print()` is another OP on the graph, which doesn't execute until you evaluate the graph. Check [this answer](https://stackoverflow.com/q/33633370/826970). — umutto, Mar 16 '18 at 07:51
I think I did, since the net produced an output. I used tf.app.run() to run my whole main function. — user3921232, Mar 16 '18 at 09:27

Timbus Calin · Answer 1 · 2020-10-17T16:41:02.473

8

UPDATED FOR TENSORFLOW 2.X

Starting from TensorFlow 2.0 (>= 2.0), since the Session object has been removed and the recommended high-level backend is Keras, the way to do get the weights is:

from tensorflow.keras.applications import MobileNetV2

model = MobileNetV2(input_shape=[128, 128, 3], include_top=False) #or whatever model
print(model.layers[0].get_weights()[0])

edited Oct 17 '20 at 16:41

answered Jan 06 '20 at 08:00

Timbus Calin

13,809
5
41
59

score 2 · Answer 2 · answered Aug 08 '18 at 15:25

I believe there are multiple issues to be tackled here.

Running eval() should be accompanied by a session. As suggested in In TensorFlow, what is the difference between Session.run() and Tensor.eval()?, .eval() expects a default session to be running which was not likely in your case earlier. So, the best option here was to precede the code with a session. From your comment, we can see that this is done.

The variables in the hidden layers (i.e. weights/kernels) need to be initialized once the graph is complete. Hence, you might want to use something similar to this:


init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
with tf.variable_scope("hidden_layer2", reuse=True):
    w = tf.get_variable("kernel")
    print(w.eval(session=sess))

I also think it is better to print a tensor anywhere outside the function using its name. Existing tensors in the graph can be obtained using `[node.name for node in tf.get_default_graph().as_graph_def().node]` and print the tensor `tf.get_default_graph().get_tensor_by_name("<>:<>").eval(session=<>)` for e.g. `tf.get_default_graph().get_tensor_by_name("const:0").eval(session=sess)`. — Bhargavi, Aug 09 '18 at 13:06

score 0 · Answer 3 · answered Mar 16 '18 at 08:30

0

Try to do this,

w = tf.get_variable("kernel")
print(w.eval())

answered Mar 16 '18 at 08:30

Darshit Mulani

31
8

Eval does not work for me. If I use your code it says "no default session registered, use eval(session=sess)", if I create a session and pass it to eval, it says "Attempting to use uninitialized value hidden_layer2/kernel" – user3921232 Mar 16 '18 at 09:08
try this import tensorflow as tf ; sess = tf.InteractiveSession() ; a = tf.print(w,[w]) ; print(a.eval()) ; #I have used semicolon to show a new line as it is a comment – Darshit Mulani Mar 16 '18 at 13:44
Thank you for the answer, but this is also not working. It says the layer, from which I want to get the weights, is not initialized yet. Maybe I need to run it somewhere else. I added the full code to my question. – user3921232 Mar 16 '18 at 18:59
I hope that someone more qualified answer and help you soon. :) Good luck. and if you find a solution please do mention here. :) – Darshit Mulani Mar 18 '18 at 18:56
Ok, I did not found any other working solution. I switched now to keras, there it works in seconds and without a problem like this for the weights of layer 0 in your model: layer0 = model.layers[0]; weights = layer0.get_weights()[0]; – user3921232 Mar 19 '18 at 19:03

score 0 · Answer 4 · answered Oct 29 '19 at 14:00

I took a different approach. First I list all the trainable variables and use the index of the required variable and run it using the current session. The code is attached below:

variables = tf.trainable_variables()

init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)

print("Weight matrix: {0}".format(sess.run(variables[0])))  # For me the variable at the 0th index was the one I required

score 0 · Answer 5 · answered Jan 06 '20 at 07:54

0

An example on how to print weights per layer in tensorflow.js:

// 
const model = tf.sequential();
...

// kernel:
model.layers[0].getWeights()[0].print()

// bias:
model.layers[0].getWeights()[1].print()

answered Jan 06 '20 at 07:54

Maksim Shamihulau

1,219
1
15
17

score 0 · Answer 6 · answered Jul 08 '20 at 19:55

As an update to Timbus Calin answer in Tensorflow 2, biases can be accessed also using get_weights(), specifically get_weights()[1].

To access and print weights and biases for example in feedforward network:

for layer in self.model.layers:
      print(layer.get_weights()[0]) # weights
      print(layer.get_weights()[1]) # biases

How to print weights in Tensorflow?

6 Answers6