1

I include Precision@K as a custom metric in Keras. According to the documentation

import keras.backend as K

def mean_pred(y_true, y_pred):
    return K.mean(y_pred)

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy', mean_pred])

I just need to calculate a function using Keras backend and pass it at the compilation step.

With Numpy only precision@k could be calculated as the following example:

def precisionatk(y_true,y_pred,k)
    precision_average = []
    idx =  (-y_pred).argsort(axis=-1)[:,:k]
    for i in range(idx.shape[0]):
        precision_sample = 0
        for j in idx[i,:]:
            if y_true[i,j] == 1:
                precision_sample += 1
        precision_sample = precision_sample / k
        precision_average.append(precision_sample)
    return np.mean(precision_average)


y_true = np.array([[0,0,1,0],[1,0,1,0]])
y_pred = np.array([[0.1,0.4,0.8,0.2],[0.3,0.2,0.5,0.1]])

print(precisionatk(y_true,y_pred,2))

0.75

So, how to I translate this to Keras backend?

EDIT: I'm working with a multilabel problem and the y_true is always an array with ones or zeros, and y_prediction each class probability.

Miguel
  • 2,738
  • 3
  • 35
  • 51
  • 1
    There is an implementation of `Percision@K` in `tf`. You can find it here https://www.tensorflow.org/api_docs/python/tf/metrics/precision_at_k. See this post to know how to use a `tf` metric in keras. https://stackoverflow.com/questions/45947351/how-to-use-tensorflow-metrics-in-keras – Sreeram TP Mar 26 '19 at 13:44
  • Thanks but I think `precision_at_k` from tf considers the top k even for the true values as well (which in the end of the day will be first k index with ones). For example, if in a prediction@k=1 I have the highest score at index 10 but in the true y there is a one is the 0 index as well in the 10th, it will calculate a precision of zero because it will consider as true value the index 0. – Miguel Mar 26 '19 at 15:16

1 Answers1

0

You can use Keras-native implementation of Precision@k: tf.keras.metrics.TopKCategoricalAccuracy

https://www.tensorflow.org/api_docs/python/tf/keras/metrics/TopKCategoricalAccuracy

Hamidreza Hosseinkhani
  • 742
  • 7
  • 10