1

Note: I already read keras forward pass with tensorflow variable as input but it did not help.

I'm training an auto-encoder unsupervised neural-network with Keras with the MNIST database:

import keras, cv2
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(60000, 784).astype('float32') / 255.0
x_test = x_test.reshape(10000, 784).astype('float32') / 255.0

model = Sequential()
model.add(Dense(100, activation='sigmoid', input_shape=(784,)))
model.add(Dense(10, activation='sigmoid'))
model.add(Dense(100, activation='sigmoid'))
model.add(Dense(784, activation='sigmoid'))
model.compile(loss='mean_squared_error', optimizer='sgd')
history = model.fit(x_train, x_train, batch_size=1, epochs=1, verbose=0)

Then I would like to get the output vector when the input vector is x_test[i]:

for i in range(100):
    x = x_test[i]
    a = model(x)
    cv2.imshow('img', a.reshape(28,28))
    cv2.waitKey(0)

but I get this error:

All inputs to the layer should be tensors.

How should I modify this code to do a forward pass of an input vector in the neural network, and get a vector in return?

Also how to get the activation after, say, the 2nd layer? i.e. don't propagate until the last layer, but get the output after the 2nd layer.
Example: input: vector of size 784, output: vector of size 10

Basj
  • 41,386
  • 99
  • 383
  • 673

2 Answers2

3

To run a model after you've finished training it you need to use keras predict(). This will evaluate the graph, given your input data. Note that the input data must be the same dimensions as the specified model inputs, which in your case looks to be [None, 784]. Keras does not require you to specify the batch dimension but you still need a 2D array going in. Do something like..

x = x_test[5]
x = x[numpy.newaxis,:]
out_val = model.predict(x)[0]

if you just want to process a single value.

The numpy.newaxis is required to make a 2D array and thus match your input size. You can skip this if you pass in an array of values to evaluate all at once.

With Keras/Tensorflow, your model is a graph/function, not standard python procedural code. You can't call it with data directly. You need to create functions and then call the functions. To get the output from an intermediate layer you can do something like..

OutFunc = K.function([model.input], [model.layers[2].output])
out_val = OutFunc([x])[0]

again, keep in mind there is a batch dimension on the input which will be produced in the output. There's a number of posts on getting data from intermediate layers if you need some additional examples. For instance see Keras, How to get the output of each layer?

bivouac0
  • 2,494
  • 1
  • 13
  • 28
  • Thanks @bivouac0, but even with `model.predict([x])`, it doesn't work (`ValueError: Error when checking input: expected dense_1_input to have shape (784,) but got array with shape (1,) `). Here my `x` is a numpy array of shape `(784, 1)`, so it is not `[None, 784]`. – Basj Nov 17 '18 at 15:14
  • I found the answer @bivouac0: when x is just a (784,1) numpy array, we should use this: `model.predict([[x]])` with a double `[[...]]`. – Basj Nov 17 '18 at 15:29
  • The key is you have to `predict` with data that is the correct shape for the network's input. Adding a bracket around the data variable simply adds a dimension of size 1 to the data. From your code above I wouldn't expect you to need 2 dimensions. What's the dimension of `x` before you input it? It should be shape=(784) since it's an image right? – bivouac0 Nov 17 '18 at 15:38
  • `print(x.shape)`: `(784,)`. With `predict([x])` it does not work. In fact `x = x_test[0]`, and x_test is defined via `x_test = x_test.reshape(10000, 784).astype('float32') / 255.0`. What do you think? – Basj Nov 17 '18 at 15:49
  • 1
    Interesting. Apparently at some point they added the ability to support multiple input models and now you need to feed `predict` a list of input data. The documentation doesn't indicate this very well but looking at their code, they're definitely iterating on the data, thus requiring the extra pair of brackets. – bivouac0 Nov 17 '18 at 16:34
0

An other way to do this than the accepted answer: when x is just a (784,) or (784,1) numpy array, we can use this: 

model.predict([[x]])

with a double [[...]].

Basj
  • 41,386
  • 99
  • 383
  • 673
  • Just be careful using the double-bracket trick as it doesn't work with older Keras versions and I wouldn't be surprised to see it change in a future version. The `numpy.newaxis` method is likely more stable. – bivouac0 Nov 18 '18 at 00:50