Custom Loss function, Keras \\ ValueError: No gradients

Question

I'm attempting to wrap my Keras neural network in a class object. I have implemented the below outside of a class setting, but I want to make this more object-friendly. To summarize, my model calls function sequential_model which creates a sequential model. Within the compile step, I have defined my own loss function weighted_categorical_crossentropy which I want the sequential model to implement. However, when I run the code below I get the following error: ValueError: No gradients provided for any variable:

I suspect the issue is with how I'm defining the weighted_categorical_crossentropy function for its use by sequential.

Again, I was able to get this work in a non-object oriented way. Any help will be much appreciated.

from tensorflow.keras import Sequential, backend as K

class MyNetwork(): 
        
    def __init__(self, file, n_output=4, n_hidden=20, epochs=3,
                 dropout=0.10, batch_size=64, metrics = ['categorical_accuracy'],
                 optimizer = 'rmsprop', activation = 'softmax'):

    [...] //Other Class attributes
 
    def model(self):
        self.model = self.sequential_model(False)
        self.model.summary()


    def sequential_model(self, val):
        K.clear_session()
        if val == False:
            self.epochs = 3
        regressor = Sequential()
        #regressor.run_eagerly = True
        regressor.add(LSTM(units = self.n_hidden, dropout=self.dropout, return_sequences = True, input_shape = (self.X.shape[1], self.X.shape[2])))
        regressor.add(LSTM(units = self.n_hidden, dropout=self.dropout, return_sequences = True))
        regressor.add(Dense(units = self.n_output, activation=self.activation))
    
        self.weights = np.array([0.025,0.225,0.78,0.020])

        regressor.compile(optimizer = self.optimizer, loss = self.weighted_categorical_crossentropy(self.weights), metrics = [self.metrics])
        regressor.fit(self.X, self.Y*1.0,batch_size=self.batch_size, epochs=self.epochs, verbose=1, validation_data=(self.Xval, self.Yval*1.0))

        return regressor

    def weighted_categorical_crossentropy(self, weights):
        weights = K.variable(weights)
        def loss(y_true, y_pred):
            y_pred /= K.sum(y_pred, axis=-1, keepdims=True)
            y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())
            loss = y_true * K.log(y_pred) * weights
            loss = -K.sum(loss, -1)
            return loss

Answer 1

There are several problems with above code, but the most noticeable one is you don't return the loss from weighted_categorical_crossentropy. It should look more like:

    def weighted_categorical_crossentropy(self, weights):
        weights = K.variable(weights)
        def loss(y_true, y_pred):
            y_pred /= K.sum(y_pred, axis=-1, keepdims=True)
            y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())
            loss = y_true * K.log(y_pred) * weights
            loss = -K.sum(loss, -1)
            return loss
        return loss # Return the callable function!

The error is ValueError: No gradients provided for any variable because the loss method doesn't return anything, it returns None! If you try to fit a method with loss=None, the model will have no way of computing gradients and therefore it will throw the same exact error.

Next up is the that you are using return_sequences = True in the layer right before a non-recurrent layer. This causes the Dense layer to be called on mis-shaped data, that's appropriate only for recurrent layers. Don't use it like that.
If you have a good reason for using the return_sequences = True, then you must add Dense layer like:

model.add(keras.layers.TimeDistributed(keras.layers.Dense(...)))

This will cause the Dense layer to act on output sequence on every time step separately. This also means that your y_true must be of appropriate shape.

There could be other problems with the custom loss function that you defined, but I can not deduce the input/output shapes, so you will have to run it and add see if it works. There will probably be matrix multiplication shape mismatch.

Last but not least, think about using the Sub-classing API. Could it make any of your operations easier to write?

Thanks for reading and I'll update this answer once I have that info. Cheers.