0

I just started with deep learning and i want to get the input/output of each layer in real-time. I am using google colab with tensorflow 2 and python 3. I tried to get the layers like this but for some reason that i don't understand is not working. Any help will be appreciated.

# Here are imports 

from __future__ import absolute_import, division, print_function, unicode_literals

try:
  # %tensorflow_version only exists in Colab.
  %tensorflow_version 2.x
except Exception:
  pass
import tensorflow as tf

from tensorflow.keras import datasets, layers, models
import matplotlib.pyplot as plt


from tensorflow.keras import backend as K



# I am using CIFAR10 dataset

(train_images, train_labels), (test_images, test_labels) = 
datasets.cifar10.load_data()

Normalize pixel values to be between 0 and 1
train_images, test_images = train_images / 255.0, test_images / 255.0

# Here is the model

model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(10, activation='softmax'))

# Compilation of the model 

model.compile(optimizer='adam',
          loss='sparse_categorical_crossentropy',
          metrics=['accuracy'])

history = model.fit(train_images, train_labels, epochs=10, 
                validation_data=(test_images, test_labels))
# Based on 
https://stackoverflow.com/questions/41711190/keras-how-to-get-the-output-of-each-layer

# I tried this 

tf.compat.v1.disable_eager_execution()
inp = model.input                                    # input placeholder
outputs = [layer.output for layer in model.layers]     # all layer outputs
functors = [K.function([inp, K.learning_phase()], [out]) for out in outputs]    # evaluation functions

Testing
test = np.random.random(input_shape)[np.newaxis,...]
layer_outs = [func([test, 1.]) for func in functors]
print(layer_outs)


#The error appear at line 
functors = [K.function([inp, K.learning_phase()], [out]) for out in outputs] 


#I got this error message
Tensor Tensor("conv2d/Identity:0", shape=(None, 30, 30, 32), dtype=float32) is not an element of this graph.
cUser
  • 392
  • 8
  • 25

1 Answers1

1

This error basically tells you that you want to change the graph after compiling it. When you call compile, TF will statically define all operations. You have to move the code snippet where you define functors above the compile method. Just swap the last lines with these ones: 

tf.compat.v1.disable_eager_execution()
inp = model.input                                    # input placeholder
outputs = [layer.output for layer in model.layers]     # all layer outputs
functors = [K.function([inp, K.learning_phase()], [out]) for out in outputs]    # evaluation functions



model.compile(optimizer='adam',
          loss='sparse_categorical_crossentropy',
          metrics=['accuracy'])

history = model.fit(train_images, train_labels, epochs=1, 
                validation_data=(test_images, test_labels))

#Testing
input_shape = [1] + list(model.input_shape[1:])
test = np.random.random(input_shape)
layer_outs = [func([test, 1.]) for func in functors]
print(layer_outs)
Lau
  • 1,353
  • 7
  • 26
  • I did as you said and i get the same error Tensor Tensor("conv2d/Identity:0", shape=(None, 30, 30, 32), dtype=float32) is not an element of this graph. – cUser Nov 12 '19 at 12:24
  • However i have one question related to this. Is there any possibility to get the output layers during the training execution ? – cUser Nov 12 '19 at 12:38
  • 1
    Yes, you should use Callbacks for that one. https://keras.io/callbacks/ – Lau Nov 12 '19 at 13:38
  • I tried to do it but something it just does not click in my mind. I created a custom callback and i tried to show output layer at the end of the batch however i still dont figure out how to make it work . DO you have some suggestion ? – cUser Nov 13 '19 at 14:43
  • Pls make a new thread with code, where you show your custom callback. You can link me and I try to help you. However, i suggest to save the data as numpy array an visualize them after training. – Lau Nov 13 '19 at 14:49
  • Okay will do, thank you :). I need them during the training phase because i want to achieve a interactive visualization during training phase – cUser Nov 13 '19 at 14:51
  • I don't know how to link you but here is the question https://stackoverflow.com/q/58844459/8861099 – cUser Nov 13 '19 at 19:45