problem defining a custom metric to calculate "geometric mean score" for "tensorflow.keras"

Question

I am working on an imabalanced classification problem in tensorflow.keras. And I decided to calculate "geometric mean score" as suggested by this answer on cross validated. I found an implementation of it in a package called imbalanced-learn and realised that it can't be used as one of the metric in tensorflow.keras.Model.compile(metrics=[]); And since I also wanted to pass an argument to it in every call, I decided to implement a custom metric myself and use it. But I got an error during testing, that said:

AttributeError: 'GeometricMeanScore' object has no attribute '_trainable'

Here is my code for custom metric:

from imblearn.metrics import geometric_mean_score
from tensorflow.keras import metrics
import numpy as np

class GeometricMeanScore(metrics.Metric):
    def __init__(self, average):
        # this 'average' is an argument "geometric_mean_score" takes for calculation.
        self.average = average
        # to store result
        self.geometric_score = 0

    # from looking at source code on github, I could learn that function that will be called for calculation is named 'update_state' and this function is what that accepts 'y_true' and 'y_pred'
    def update_state(self, y_true, y_pred):
        # store the result
        self.geometric_score = geometric_mean_score(y_pred=y_pred, y_true=y_true, average=self.average)

    def result(self):
        # access/print the result during every batch of every epoch.
        return self.geometric_score

testing it:

# creating an instance
abc = GeometricMeanScore(average='weighted')
abc.update_state(y_true=np.array([0, 1, 2, 0, 1, 2]), y_pred=np.array([0, 2, 1, 0, 0, 1]))
print(abc.result())

complete error:

C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python38\python.exe "C:/Users/neevaN_Reddy/Documents/custom_metric/defining custom meric.py"
2020-07-22 12:09:24.916554: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2020-07-22 12:09:24.916874: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
  File "C:/Users/neevaN_Reddy/Documents/custom_metric/defining custom meric.py", line 19, in <module>
    abc.update_state(y_true=[0, 1, 2, 0, 1, 2], y_pred=[0, 2, 1, 0, 0, 1])
  File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\utils\metrics_utils.py", line 80, in decorated
    for weight in metric_obj.weights:
  File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1114, in weights
    return self.trainable_weights + self.non_trainable_weights
  File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1080, in trainable_weights
    if self.trainable:
  File "C:\Users\neevaN_Reddy\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1007, in trainable
    return self._trainable
AttributeError: 'GeometricMeanScore' object has no attribute '_trainable'

Process finished with exit code 1

What am I missing and how do I fix this error and use it in tf.keras like this:

tensorflow.keras.Model.compile(metrics=[GeometricMeanScore(average='weighted')])

Answer 1

The function geometric_mean_score() takes NumPy-arrays as input. However keras will pass tensors to your custom metric function.

Here and here are related posts.

This is a blog post on custom segmentation loss functions and their implementation in TensorFlow. It might help and guide you on how to write a custom metric / loss function.