1

This is probably quite a basic tensorflow/keras question but it is I can't seem to find it in the docs. I'm looking to retrieve the output of the hidden layer as numerical values for use in subsequent calculations. Here's the model

from io import StringIO
import pandas as pd
import numpy as np
import keras

data_str = """     
ti,z1,z2
0.0,1.000,0.000
0.1,0.606,0.373
0.2,0.368,0.564
0.3,0.223,0.647
0.4,0.135,0.669
0.5,0.082,0.656
0.6,0.050,0.624
0.7,0.030,0.583
0.8,0.018,0.539
0.9,0.011,0.494
1.0,0.007,0.451"""
data = pd.read_csv(StringIO(data_str), sep=',')

wd = r'/path/to/working/directory'
model_filename = os.path.join(wd, 'example1_with_keras.h5')

RUN = True
if RUN:
    model = keras.Sequential()
    model.add(keras.layers.Dense(3, activation='tanh', input_shape=(1, )))
    model.add(keras.layers.Dense(2, activation='tanh'))
    model.compile(optimizer='adam', loss='mean_squared_error')
    model.fit(data['ti'].values, data[['z1', 'z2']].values, epochs=30000)
    model.save(filepath=model_filename)
else:
    model = keras.models.load_model(model_filename)

outputs = model.layers[1].output
print(outputs)

This prints the following:

>>> Tensor("dense_2/Tanh:0", shape=(?, 2), dtype=float32)

How can I get the output as a np.array rather than a Tensor object?

CiaranWelsh
  • 7,014
  • 10
  • 53
  • 106

3 Answers3

2

Using model.layer[1].output does not produce an output, it simply returns the tensor definition of the output. In order to actually produce an output, you need to run your data through the model and specify model.layer[1].output as the output.

You can do this by using tf.keras.backend.function (documentation), which will return Numpy arrays. A similar question to yours can be found here.

The following should work for your example if you only want the output from model.layers[1].output and if you convert your data to a Numpy array for input:

from keras import backend as K
outputs = [model.layers[1].output]
functor = K.function([model.input, K.learning_phase()], outputs)
layer_outs = functor([data, 1.])
Luke DeLuccia
  • 541
  • 6
  • 16
  • 1
    Perfect. This is what I'm looking for as it allows me to get the output of any layer, not just the final output layer. Just note that I had to modify this code to iterate over the input such that the hidden layer outputs are computed one at a time - i.e. `[functor([numpy.array([[i]]), 1.]) for i in data['ti'] ]`. Thanks – CiaranWelsh Jan 08 '19 at 09:09
2

What you want is just:

model.predict(inputs)

Which will do a forward pass of the model, given an input, and produce numerical outputs.

Dr. Snoopy
  • 55,122
  • 7
  • 121
  • 140
  • Hi Matias, thanks for the response. I am however looking for the output of the *hidden layer, rather than output of the entire model. – CiaranWelsh Jan 08 '19 at 08:55
1

As Luke already mentioned you seem to confuse what keras is actually doing here for you.

There are two phases in libraries like Keras, tensorflow or PyTorch.

1. Computational Graph inference

2. Computation using sessions

You are in phase 1 where you create a static computational graph. This does not do any computation yet but it is helpful because you know beforehand how to run data forward and backward through your graph thus making it faster than computing it every time you pass data.

If you actually want to get an output in form of a numpy-array you will have to pass data to the inputs of your graph. In Tensorflow this has to be done using sessions, but Keras hides this from you and lets you input data freely.

In keras you usually do something like scores = model.predict(X)

Jonathan R
  • 3,652
  • 3
  • 22
  • 40