Invalid tape state error in Keras due to custom metric function

Question

So, I'm having some troubles trying to implement a SSIM-based metric function in Keras.

My metric function is:

@tf.function
def custom_ssim(y_actual, y_predicted):

    y_pred_aux = tf.argmax(y_predicted, axis=-1)
    y_pred_aux = tf.expand_dims(y_pred_aux, axis=3)
    y_pred_aux = tf.cast(y_pred_aux, np.float64)

    y_actual_aux = tf.argmax(y_actual, axis=-1)
    y_actual_aux = tf.expand_dims(y_actual_aux, axis=3)
    y_actual_aux = tf.cast(y_actual_aux, np.float64)

    return 1 - tf.image.ssim(y_actual_aux, y_pred_aux, max_val=7)

The error message that I receive when trying to compile my model is the following:

InternalError: Invalid tape state.

I've already tried not to use the @tf.function decorator, only to get the following error message:

ValueError: No gradients provided for any variable: ['conv0/kernel:0', 'conv0/bias:0', 'conv1/kernel:0', 'conv1/bias:0', 'conv2/kernel:0', 'conv2/bias:0', 'conv3/kernel:0', 'conv3/bias:0', 'conv4/kernel:0', 'conv4/bias:0', 'deconv0/kernel:0', 'deconv0/bias:0', 'deconv1/kernel:0', 'deconv1/bias:0', 'deconv2/kernel:0', 'deconv2/bias:0', 'deconv3/kernel:0', 'deconv3/bias:0', 'deconv4/kernel:0', 'deconv4/bias:0'].

Trying to convert the Tensors in the function to a NumPy array with .numpy() also didn't work.

I'm using Tensorflow-GPU 2.0 and Python 3.6.

Answer 1

You get this error when you pass only the training data and missed to pass the labels in model.fit(). I was able to recreate your error using below code. You can download the dataset I am using in the program from here.

Code to recreate the issue -

%tensorflow_version 2.x
# MLP for Pima Indians Dataset saved to single file
import numpy as np
from numpy import loadtxt
import tensorflow as tf
print(tf.__version__)
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

# load pima indians dataset
dataset = np.loadtxt("/content/pima-indians-diabetes.csv", delimiter=",")

# split into input (X) and output (Y) variables
X = dataset[:,0:8]
Y = dataset[:,8]

# define model
model = Sequential()
model.add(Dense(12, input_dim=8, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

# compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Model Summary
#model.summary()

# Fit the model
model.fit(X, epochs=150, batch_size=10, verbose=0)

Output -

2.2.0
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-7ddca8f2992e> in <module>()
     28 
     29 # Fit the model
---> 30 model.fit(X, epochs=150, batch_size=10, verbose=0)

10 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    966           except Exception as e:  # pylint:disable=broad-except
    967             if hasattr(e, "ag_error_metadata"):
--> 968               raise e.ag_error_metadata.to_exception(e)
    969             else:
    970               raise

ValueError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
        outputs = self.distribute_strategy.run(
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
        return fn(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:541 train_step  **
        self.trainable_variables)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1804 _minimize
        trainable_variables))
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:521 _aggregate_gradients
        filtered_grads_and_vars = _filter_grads(grads_and_vars)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:1219 _filter_grads
        ([v.name for _, v in grads_and_vars],))

    ValueError: No gradients provided for any variable: ['dense_5/kernel:0', 'dense_5/bias:0', 'dense_6/kernel:0', 'dense_6/bias:0', 'dense_7/kernel:0', 'dense_7/bias:0'].

Solution - Pass the training labels in model.fit() and your error will be fixed.

Modified,

model.fit(X , epochs=150, batch_size=10, verbose=0)

to

model.fit(X , Y, epochs=150, batch_size=10, verbose=0)

Code -

%tensorflow_version 2.x
# MLP for Pima Indians Dataset saved to single file
import numpy as np
from numpy import loadtxt
import tensorflow as tf
print(tf.__version__)
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

# load pima indians dataset
dataset = np.loadtxt("/content/pima-indians-diabetes.csv", delimiter=",")

# split into input (X) and output (Y) variables
X = dataset[:,0:8]
Y = dataset[:,8]

# define model
model = Sequential()
model.add(Dense(12, input_dim=8, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

# compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Model Summary
#model.summary()

# Fit the model
model.fit(X , Y, epochs=150, batch_size=10, verbose=0)

Output -

2.2.0
<tensorflow.python.keras.callbacks.History at 0x7f9208433eb8>

If still not fixed, then please share the reproducible code for the error. Would be happy to help.

Hope this answers your question. Happy Learning.