1

I'm trying to understand how tf.keras.metrics.AUC(multi_label=True) works. From the docs, I'm led to understand that when working with multi-label vectors, each class is computed individually, then averaged.

However, I can't seem to get the following trivial case to compute correctly. That is, if the prediction is the same as the expected vector, why is the output not 1.0?

y_true = [
    [1, 0, 0, 0, 1],
]

acc = tf.keras.metrics.AUC(multi_label=True, num_labels=5)

acc.reset_state()
acc.update_state(tf.constant(y_true), tf.constant(y_true))
acc.result().numpy()

>>> 0.0
rodrigo-silveira
  • 12,607
  • 11
  • 69
  • 123
  • 1
    Not sure exactly why this is happening, most likely has something to do with this line from the [docs](https://www.tensorflow.org/api_docs/python/tf/keras/metrics/AUC) `For a best approximation of the real AUC, predictions should be distributed approximately uniformly in the range 0, 1. The quality of the AUC approximation may be poor if this is not the case`. – o-90 Sep 08 '22 at 14:46

0 Answers0