0

I am performing regression learning using an Artifical neural network in keras with tensorflow backend.

My input shape is [100,3] ([no of samples,no of features]).
My output shape is [100,3] ([no of samples,no of outputs]).

I want to pass a 4 dimensional data alongside each input sample (shape of this data will be [100,4]) so that i can access it for writing a custom loss function. I don't want it to get involved in the training process.

Something like this:

def wrapper(input_tensor):
    def custom_loss(y_true, y_pred):
        return binary_crossentropy(y_true, y_pred) + mean(last 4 elements of the input_tensor)
    return custom_loss

I researched quite a lot on functional API and how keras can be used with multiple inputs and multiple outputs for training the network. But, since i don't want it to get involved in the training phase and still needs to be passed onto the custom loss function - i don't think it will serve my purpose.

My intuition for solving this:
1. Append the 4 dimensional data to the input, mask those 4 input neurons while training the network, by passing only a part of the input layer(excluding the last 4 elements) to the next layer. - Problem is, i don't think we can mask neurons like this.
2. Have the 4 dimensional data as additional input to the network using functional APIs. - Problem is, i can't pass the 4 dimensional data to the custom loss function without involving it in the training process.

Can anyone please help me solve this? Please let me know, if any extra information is needed.

I think, even this question - How to use part of inputs for training but rest for loss function in Keras follows my need. But, it is not answered :(

frogatto
  • 28,539
  • 11
  • 83
  • 129
Pradeep Gowda
  • 33
  • 5

1 Answers1

1

This can easily be done through the functional API of Keras. For example:

from tensorflow.keras import Model
from tensorflow.keras.layers import *

input_1 = Input((3,))
input_2 = Input((4,))

output = Dense(100)(input_1)
output = Dense(100)(output)

model = Model(inputs=[input_1, input_2], outputs=[output, input_2])
model.summary()

Notice that I'm defining input_2, but not using it as a parameter to any layer. I'm directly passing it to the outputs parameter of Model.

The summary for this model is:

Model: "model_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_3 (InputLayer)            [(None, 3)]          0                                            
__________________________________________________________________________________________________
dense_1 (Dense)                 (None, 100)          400         input_3[0][0]                    
__________________________________________________________________________________________________
dense_2 (Dense)                 (None, 100)          10100       dense_1[0][0]                    
__________________________________________________________________________________________________
input_4 (InputLayer)            [(None, 4)]          0                                            
==================================================================================================
Total params: 10,500
Trainable params: 10,500
Non-trainable params: 0
__________________________________________________________________________________________________
Susmit Agrawal
  • 3,649
  • 2
  • 13
  • 29
  • Thank you for the response. I will try it. – Pradeep Gowda Jan 25 '20 at 17:32
  • can you please specify how to pass input_2 to custom loss function? – Pradeep Gowda Jan 27 '20 at 14:03
  • The `y_pred` value in the loss function will be an array containing `output` and `input_2` – Susmit Agrawal Jan 28 '20 at 03:30
  • Thank you for the response. However, i figured it out that there are multiple ways of passing it to the custom loss function. If we want to use the model for inference - we should use a lambda dummy layer in between for input_2 and input_3. All in all i am satisfied with your answer. – Pradeep Gowda Jan 29 '20 at 18:20