Is it possible to calculate an equal error rate(EER) for a multi-class classification problem? I'm working on a biometric user authentication problem.
If yes, can someone please provide me with some information on how to calculate it? If not, please provide some alternatives to EER?
Your question is related to this one: ROC for multiclass classification , since EER (equal error rate) is calculated from the ROC by adjusting acceptance threshold.
I will review the conceptual process of what is EER for multiclass classification.
Suppose you have n > 2 classes, for example: A, B, C and set of samples x in X with their true labels. The idea is to binarize the problem by converting it to n binary classification problems: for each class (say A) and sample (say x) there are 2 possibilities: x is in A, or x is not in A. If a in A is classified in class A, this is a true positive, if a is classified in B or C, it is false rejection or false negative. Similarly, if b in B is classified in A, it is a false acceptance or false positive. Then for each class you can compute FAR (false acceptance rate) and FRR (false rejection rate), adjust thresholds and compute EER for each class. Then you can average over the EERs of each classes. Another approach is to calculate FRR and FAR, first average them and then adjust the parameters so average FRR and average FAR are the same (this is more complicated).
An introduction for the "One vs. Rest" strategy can be found in TowardsDataScience and Data Science Stack Exchange.
Open issues:
sklearn probably handles it in the background when plotting the ROC.I hope it helps, and I will be happy to see comments and additions which will make this issue clearer and answer the open issues I wrote above.
