0

I've been using Keras for binary classification with Tensorflow backend in Python. My model is created like this : 

model = Sequential()
model.add(Dense(1000, input_dim=168319))
model.add(Dense(units=1, activation='sigmoid'))
model.compile(loss="binary_crossentropy",
          optimizer="adam",
          metrics=['accuracy'])

And my result after training looks like this : 

342/1342 [==============================] - 79s 59ms/step - loss: 0.0586 - acc: 0.9911 - val_loss: 0.4632 - val_acc: 0.8169

If I use my network to predict a sample, it gives me a number between [0,1], as it should, since I'm using a sigmoid activation function for my output neuron. One example of what my output looks like after a prediction for 6 samples which should be of class 1 : 

[[1.        ][1.        ][0.99997437][0.18694757][0.18712251][0.9491884 ]]

Since the results are all floats between 0 and 1 , I've been wondering how Keras calculates the validation accuracy. Validation accuracy is measured with testing samples that are not used for training. As mentioned here how does Keras compute validation accuracy and training accuracy?, the validation accuracy is calculated with (amount of correct guesses)/(total amount of guesses) .

My Question here is, how does Keras decide which class the guess belongs to, when is it "correct". Does it round up at 0.5? Or is everything between 0 and 1 classified as "wrong guess" (Would expect a way lower number in validation accuracy then)?

MWeb
  • 13
  • 4
  • Since you are using `binary_crossentropy`, in this case each of your six classes is evaluated separately. For each a value above 0.5 is set to 1. Below 0.5 is set to 0. If you were using `categorical_crossentropy` then only one of the classes can be 1. Whichever has the highest probability will be set to 1. The rest will be set to 0. If you think about it, the `binary` case is just a special case of the `categorical` one. The highest value is set to 1. So if and only if the prediction is bigger than 0.5 then the prediction is set to 1. – Karl Dec 07 '18 at 14:36
  • Thanks for answering so quickly! Do you have any references for Keras rounding up at 0.5? – MWeb Dec 07 '18 at 14:48
  • https://github.com/keras-team/keras/blob/master/keras/metrics.py. See `binary_accuracy` – Karl Dec 07 '18 at 15:04
  • Thank you! If you want, you can put that into the answer and I'll accept it. – MWeb Dec 17 '18 at 09:15

1 Answers1

0

Since you are using binary_crossentropy, in this case each of your six classes is evaluated separately. For each a value above 0.5 is set to 1. Below 0.5 is set to 0. If you were using categorical_crossentropy then only one of the classes can be 1. Whichever has the highest probability will be set to 1. The rest will be set to 0. If you think about it, the binary case is just a special case of the categorical one. The highest value is set to 1. So if and only if the prediction is bigger than 0.5 then the prediction is set to 1.

For more info, see binary_accuracy under github.com/keras-team/keras/blob/master/keras/metrics.py.

Karl
  • 5,573
  • 8
  • 50
  • 73