78

I have come across few (Machine learning-classification problem) journal papers mentioned about evaluate accuracy with Top-N approach. Data was show that Top 1 accuracy = 42.5%, and Top-5 accuracy = 72.5% in the same training, testing condition. I wonder how to calculate this percentage of top-1 and top-5?

Can some one show me example and steps to calculate this?

Thanks

D_9268
  • 1,039
  • 2
  • 9
  • 17
  • 1
    @SalvadorDali check below answer from "rcpinto". Maybe something you can understand also from there. Thanks for your reply, i will make my question clear next time. – D_9268 Jun 08 '16 at 07:29

2 Answers2

168

Top-1 accuracy is the conventional accuracy: the model answer (the one with highest probability) must be exactly the expected answer.

Top-5 accuracy means that any of your model 5 highest probability answers must match the expected answer.

For instance, let's say you're applying machine learning to object recognition using a neural network. A picture of a cat is shown, and these are the outputs of your neural network:

  • Tiger: 0.4
  • Dog: 0.3
  • Cat: 0.1
  • Lynx: 0.09
  • Lion: 0.08
  • Bird: 0.02
  • Bear: 0.01

Using top-1 accuracy, you count this output as wrong, because it predicted a tiger.

Using top-5 accuracy, you count this output as correct, because cat is among the top-5 guesses.

rcpinto
  • 4,166
  • 2
  • 24
  • 25
  • 9
    Thanks for this answer. In your opinion, is Top-5 really a good metric, or is it a way to exaggerate the true capabilities of a neural network? If I were blind, and asked someone to tell me what animal was in front of me, I would expect "It is a cat" rather than "It is either a Tiger, Dog, Cat, Lynx, or Lion". – Jonathon Reinhart Dec 05 '17 at 14:09
  • 7
    I think top-5 metric is useful, among other reasons, because a picture can have more than one object... – cag51 Apr 29 '18 at 22:59
  • 6
    So we can say that top-5 accuracy will always be higher than Top-1 accuracy – Ashiq Imran May 31 '18 at 20:02
  • 5
    Higher or equal – NicolasElPapu May 15 '20 at 20:29
  • 4
    @JonathonReinhart Might be a little late, but most large scale models are trained on the ImageNet dataset, containing 1000 possible classes, so in reality an inference on how well the model performs at the Top-5 is highly realistic as it narrows out the other 995 possible classes. However if you only have 10 classes then Top-5 is not useful. – notMyName Dec 29 '21 at 15:57
5

The Complement of the accuracy is the error, The top-1 error is the percentage of time that the classifier did not give the correct class highest probability score. The top-5 error:- The percentage of time that the classifier did not include the correct class among the top 5 probabilities or guesses.

Mohammed Awney
  • 1,292
  • 16
  • 21