how to see tensor value of a layer output in keras

Question

I have a Seq2Seq model. I am interested to print out the matrix value of the output of the encoder per iteration.

So for example as the dimension of the matrix in the encoder is (?,20) and the epoch =5 and in each epoch, there are 10 iteration, I would like to see 10 matrix of the dimension (?,20) per epoch.

I have gone to several links as here but it still does not print out the value matrix. With this code as mentioned in the aboved link:

import keras.backend as K

k_value = K.print_tensor(encoded)
print(k_value)

I got:

Tensor("Print:0", shape=(?, 20), dtype=float32)

Is there any straightforward way of showing the tensor value of each layer in Keras?

Update 1

by trying this code: K_value = K.eval(encoded) it raises this error:

Traceback (most recent call last):
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1278, in _do_call
    return fn(*args)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1263, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1350, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'input' with dtype float and shape [?,45,50]
     [[Node: input = Placeholder[dtype=DT_FLOAT, shape=[?,45,50], _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
     [[Node: encoder_lstm/add_16/_25 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_460_encoder_lstm/add_16", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/sgnbx/Downloads/projects/LSTM_autoencoder/justfun.py", line 121, in <module>
    k_value = K.eval(encoded)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 671, in eval
    return to_dense(x).eval(session=get_session())
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 680, in eval
    return _eval_using_default_session(self, feed_dict, self.graph, session)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 4951, in _eval_using_default_session
    return session.run(tensors, feed_dict)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 877, in run
    run_metadata_ptr)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1100, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1272, in _do_run
    run_metadata)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1291, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'input' with dtype float and shape [?,45,50]
     [[Node: input = Placeholder[dtype=DT_FLOAT, shape=[?,45,50], _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
     [[Node: encoder_lstm/add_16/_25 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_460_encoder_lstm/add_16", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Caused by op 'input', defined at:
  File "/home/sgnbx/Downloads/projects/LSTM_autoencoder/justfun.py", line 113, in <module>
    inputs = Input(shape=(SEQUENCE_LEN, EMBED_SIZE), name="input")
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/keras/engine/input_layer.py", line 177, in Input
    input_tensor=tensor)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/keras/engine/input_layer.py", line 86, in __init__
    name=self.name)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 515, in placeholder
    x = tf.placeholder(dtype, shape=shape, name=name)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py", line 1735, in placeholder
    return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 4925, in placeholder
    "Placeholder", dtype=dtype, shape=shape, name=name)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
    return func(*args, **kwargs)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
    op_def=op_def)
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input' with dtype float and shape [?,45,50]
     [[Node: input = Placeholder[dtype=DT_FLOAT, shape=[?,45,50], _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
     [[Node: encoder_lstm/add_16/_25 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_460_encoder_lstm/add_16", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Exception ignored in: <bound method BaseSession.__del__ of <tensorflow.python.client.session.Session object at 0x7fd900525c50>>
Traceback (most recent call last):
  File "/home/sgnbx/anaconda3/envs/py3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 686, in __del__
TypeError: 'NoneType' object is not callable

Process finished with exit code 1

@Anakin Thank you so much for following my question, Do you mind to give an example of how can I do that? — sariii, May 22 '19 at 18:21

Thibault Bacqueyrisses · Answer 1 · 2019-05-22T20:02:17.503

4

Very simple way to print a tensor :

from keras import backend as K

k_value = K.eval(tensor)
print(k_value)

UPDATE 1

Create a callback to print at the end of each epoch :

class callback(Callback):
    def __init__(self, model, X_train):
        self.model = model
        self.x = X_train

    def on_train_begin(self, logs={}):
        return

    def on_train_end(self, logs={}):
        return

    def on_epoch_begin(self, epoch, logs={}):
        return

    def on_epoch_end(self, epoch, logs={}):
        inp = model.input  # input placeholder
        outputs = model.layers[N].output  # get output of N's layer
        functors = K.function([inp, K.learning_phase()], [outputs])
        layer_outs = functors([self.x, 1.])
        print('\r OUTPUT TENSOR : %s' % layer_outs)

        return

    def on_batch_begin(self, batch, logs={}):
        return

    def on_batch_end(self, batch, logs={}):
        return

Call this function in your fit() method like that :

callbacks=[callback(model = model, X_train = X_train)])

Inspired from Keras, How to get the output of each layer?

Hope this will finally help you !

edited May 22 '19 at 20:02

answered May 22 '19 at 18:21

Thibault Bacqueyrisses

2,281
1
6
18

thanks for your answer. but which library is tensor here? this code written in KERAS – sariii May 22 '19 at 18:29
Can you also include the import libraries, please? if you mean the tensor array that I have I have already tried this(a variation of this) but it raises error, if you mean tensor= my tensor array I can update the question with the error which raises. – sariii May 22 '19 at 18:39
Thank you gaian for following, I did not mean that I have the restriction for using only Keras. the thing is that this updated answer and also the previous one raises anerror, I update my question with the error – sariii May 22 '19 at 18:46
I'm not sure, but it's look like your tensor is empty. If you try to print this tensor : tensor = K.ones(shape=(3, 4, 5)) with my code, is it working ? – Thibault Bacqueyrisses May 22 '19 at 18:55
I have my final output printed, So it means this code is working fine. I need the output in each iteration. – sariii May 22 '19 at 18:59
I think I got what is happening here, I am trying to print out before fitting the model in the process of building the model. on the other hand, if I print it out after fitting the model is it is getting the same error as above. In case I create a model out of my encoder and use that model with the code above again the same error! – sariii May 22 '19 at 19:10
I think you need to create a callback that will print the tensor a each epoch, and give to it the tensor you want – Thibault Bacqueyrisses May 22 '19 at 19:12
Thanks again, do you have any similar example for calling the callbacks on the tensor? – sariii May 22 '19 at 19:14
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/193788/discussion-between-thibault-bacqueyrisses-and-sariii). – Thibault Bacqueyrisses May 22 '19 at 19:34

how to see tensor value of a layer output in keras

1 Answers1

Linked