0

I wrote this function that takes in an image and generates a prediction. The level of confidence for the prediction that the function reports is greater than 100% many times. Sometimes the prediction is correct and reports a high level of confidence. Sometimes it is incorrect and still reports a high level of confidence. Can you please help me identify the error in my level of confidence code?

Final line of model structure:

outputs = tf.keras.layers.Dense(3)(x)
user17020095
  • 41
  • 1
  • 7

1 Answers1

1

If you want your output to be between 0 and 1, you should use either a 'sigmoid' or 'softmax' activation in your last layer:

outputs = tf.keras.layers.Dense(3, activation='softmax')(x)

Careful, however, because softmax output can't really be interpreted as probability.

Nicolas Gervais
  • 33,817
  • 13
  • 115
  • 143
  • 3
    Why can't softmax output be interpreted as probability? – Dave Nov 16 '21 at 14:34
  • Thank you Nicolas. What can I use to generate an accurate probability? – user17020095 Nov 16 '21 at 14:37
  • 2
    Softmax probabilities are uncalibrated but they are still probabilities. – Dr. Snoopy Nov 16 '21 at 14:37
  • @Dr.Snoopy Why are softmax probabilities uncalibrated? I don't think this is an inherent issue in all softmax regression e.g., multinomial logistic regression. (Binary logistic regression has good calibration.) – Dave Nov 16 '21 at 14:39
  • Can you please explain what you mean by uncalibrated? How can I generate a calibrated probability? – user17020095 Nov 16 '21 at 14:39
  • 1
    @user17020095 That's really a question to as on [Cross Validated](https://stats.stackexchange.com). – Dave Nov 16 '21 at 14:40
  • 1
    Read this paper about the topic: https://arxiv.org/abs/1706.04599 – Dr. Snoopy Nov 16 '21 at 14:40
  • Thank you @Dave and @Dr. Snoopy! @Nicolas I see " UserWarning: "`sparse_categorical_crossentropy` received `from_logits=True`, but the `output` argument was produced by a sigmoid or softmax activation and thus does not represent logits. Was this intended?" What is the relationship between softmax and logits? https://stackoverflow.com/questions/41455101/what-is-the-meaning-of-the-word-logits-in-tensorflow I see that logits help map probabilities. – user17020095 Nov 16 '21 at 14:52
  • 1
    You will indeed need to remove `from_logits=True`. – Nicolas Gervais Nov 16 '21 at 15:12