Custom Metrics Keras

Question

i need help to create a custom metrics in keras. I need to count how many times my error is equal to zero (y_pred - y_true = 0).

I tried this:

n_train = 1147 # Number of samples on training set
c = 0 # Variable to count
def our_metric(y_true, y_pred):
  if y_true-y_pred == 0:
    c += 1
  return c/n_train

But i'm getting this error:

OperatorNotAllowedInGraphError: using a tf.Tensor as a Python bool is not allowed in Graph execution. Use Eager execution or decorate this function with @tf.function.

EDIT: Using the solution proposed here:

Creating custom conditional metric with Keras

I solved my problem as this:

c = tf.constant(0)
def our_metric(y_true, y_pred):
  mask = K.equal(y_pred, y_true) # TRUE if y_pred = y_true
  mask = K.cast(mask,K.floatx())
  s = K.sum(mask)
  return s/n_train

I used the solution proposed here: https://stackoverflow.com/questions/51902088/creating-custom-conditional-metric-with-keras?newreg=e11e214e50b24041bf97b1688c7ec16a — Murilo Souza, Nov 07 '19 at 15:21

score 0 · Answer 1 · answered Nov 07 '19 at 13:52

You can't run Python comparison in plain tensorflow (using static graphs).

You have to enable eager mode, a wrapper which let's you use some Python control statements (like if or loop). Just decorate your function as the error suggests or issue tf.enable_eager_execution() at the beginning of your script.

You may also want to update your code to use tf2.0, it's more intuitive and has eager mode on by default.

score 0 · Answer 2 · answered Nov 07 '19 at 15:22

There are numerous ways to use Keras backend functions to count the number of times a value is equal to zero. You just have to think a bit outside of the box. Here is an example:

diff = y_true - y_pred
count = K.sum(K.cast(K.equal(diff, K.zeros_like(diff)), 'int8'))

There's also a tf.count_nonzero operation that could be used, but mixing keras and explicit tensorflow can cause issues.

Custom Metrics Keras

2 Answers2