How to use numpy functions on a keras tensor in the loss function?

Question

I'm using Keras with TensorFlow backend to build and run a neural network. I need to use a numpy function on my output tensor in the loss function. More specifically, my loss function involves finding nearest neighbors, and I need to use the Keras functionality for ckdTree for this purpose. I have tried converting my output tensor to a numpy array using K.eval(). However, this throws an InvalidArgument error when I try to compile the model, I believe, since you can't run eval() on a symbolic variable.

Here's a toy code snippet that reproduces this error.

import numpy as np
from keras import backend as K
from keras.models import Sequential
from keras.layers.core import Flatten, Dense, Reshape
from keras.optimizers import Adam

def loss(y_true, y_pred):

    y_pred_numpy = K.eval(y_pred)
    # perform some numpy operations on y_pred_numpy
    return K.constant(0)

''' Model '''

input_shape = (10,10,10,3)
train_images = np.zeros((1,10,10,10,3))
train_labels = np.zeros((1,1,1,1,3))

model = Sequential()
model.add(Flatten(input_shape=input_shape))
model.add(Dense(3000, use_bias=True, bias_initializer='zeros'))
model.add(Reshape((10,10,10,3)))
model.summary()

opt = Adam(lr=1E-4)
model.compile(optimizer=opt, loss=loss)

The above gives the following error:

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'flatten_3_input' with dtype float
     [[Node: flatten_3_input = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
     [[Node: reshape_3/Reshape/_11 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_20_reshape_3/Reshape", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

How then do I work with Keras tensors without having to rewrite (complex) numpy functionality using Keras?

Is it really a problem in the loss function??? Have you tried compiling with `loss='mse'`? -- Even though, I'm not sure keras will accept numpy operations with the tensors. :( — Daniel Möller, May 17 '17 at 12:23
Did you have a look at [this question](https://stackoverflow.com/questions/39921607/how-to-make-a-custom-activation-function-with-only-python-in-tensorflow) already? — Mr Tsjolder from codidact, Jul 30 '17 at 18:23

score 2 · Answer 1 · answered May 17 '17 at 12:57

The direct using of this numpy function is impossible - as it's not implemented in neither Tensorflow nor Theano. Moreover - there is no a direct correspondence between tensors and arrays. Tensors should be understood as an algebraic variables whereas numpy arrays as numbers. tensor is an abstract thing and applying a numpy functions to it is usually impossible.

But you could still try to reimplement this function on your own using keras.backend functions. Then you'll use the valid tensor operations and no problem should be raised.

https://www.tensorflow.org/versions/r1.14/api_docs/python/tf/py_func or https://www.tensorflow.org/api_docs/python/tf/numpy_function provides a way to use functions that transform numpy arrays into numpy arrays on tensors. — simon, Oct 16 '19 at 23:07

How to use numpy functions on a keras tensor in the loss function?

1 Answers1