-2

I have couple of models that do binary classification and I use sklearn's predict_proba function to find the confidence scores.

predict_proba does sum up to 1 for Decision Tree, SVM, Random forest.

But when it's used with Gaussian NB it does not sum up to 1.

nb- predict_proba results

Same thing occurs in Nearest Neighbour too.

Do you have any idea why this happens and what can be done to find confidence scores?

Anwarvic
  • 12,156
  • 4
  • 49
  • 69
Burak
  • 35
  • 6
  • Because Naive Bayes is a generative model while SVM and Decision trees are both discriminative models. In other words, SVM and decision tree learn how to differentiate between the two classes. While Naive Bayes learns how to acknowledge these two classes. – Anwarvic Jun 20 '20 at 13:31
  • thank you. Is there any way that I can test the confidence score of Naive Bayes – Burak Jun 20 '20 at 16:34
  • not that I know of – Anwarvic Jun 20 '20 at 16:34
  • Are you sure they don't (within numeric accuracy)? I suggest you have a closer look - plus, please do **not** post images of code or data, copy and paste the relevant stuff in the question as **text** (in which case, a quantitative answer would be possible). – desertnaut Jun 21 '20 at 00:56

1 Answers1

0

I think if you look closely, you'll see that the predicted probabilities for the two classes do add up to ~1, accounting for rounding errors and numeric approximations in the NB process. For instance for each row where one class is ~= 1.00000000, the other class value is vanishingly small.

Second, it is hard to understand what you want from a "confidence" in this case. Many of the classification tools in sklearn have a decision_function() method that will give you a distance of the estimate from the decision boundary. In this case, the decision boundary is a function of the kind of NB you chose (Gaussian). I think this explanation of plotting a boundary might be very helpful.

But a more thorough search of SO would have yielded you this question, which I think might get closer to what you're looking for.

Savage Henry
  • 1,990
  • 3
  • 21
  • 29