Multi-class classification find probability of all classes

Question

I took an example from the keras.

https://github.com/keras-team/keras/blob/master/examples/pretrained_word_embeddings.py

sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
x = Conv1D(128, 5, activation='relu')(embedded_sequences)
x = MaxPooling1D(5)(x)
x = Conv1D(128, 5, activation='relu')(x)
x = MaxPooling1D(5)(x)
x = Conv1D(128, 5, activation='relu')(x)
x = GlobalMaxPooling1D()(x)
x = Dense(128, activation='relu')(x)
preds = Dense(len(labels_index), activation='softmax')(x)

model = Model(sequence_input, preds)
model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['acc'])

The model predicted classes probabilities which are lesser than 0. I know softmax will sum it to 1. The only one output i can see in probability using np.argmax(pre) I want the other classes probabilities at least readable.

Prediction output:
    [2.8300792e-06 4.5637703e-03 7.2316222e-02 6.7710824e-02 5.2243233e-01
     3.7763064e-04 1.2326813e-02 2.9277834e-01 4.1662962e-03 1.0876421e-05
     2.3830748e-06 1.3590348e-04 2.3074823e-02 3.3520879e-05 4.0551484e-05
     1.9896568e-06 1.0994432e-05 4.7518920e-06 2.3408763e-06 6.7659844e-06]

All of them are yielding the lesser than 0 as probability. When I use np.argmax I got 4 . How do I get probability results above 0? Instead softmax which activation should i use to get more positive probabilities?

I don't understand your question. All your predicted classes probabilities are greater than `0`. `5.2243233e-01 = 0.52243233` and `6.7710824e-02 = 0.067710824`. — giser_yugang, Jun 28 '19 at 03:37
The numbers are in scientific notation, for example 2.83e-6 = 2.83 x 10^(-6), none of these numbers are negative. — Dr. Snoopy, Jun 28 '19 at 05:56
I get it. I'm lack in formatting this numpy result. Thanks for pointing it.. 6.7710824e-02 = 0.067710824 How did you get it>? — Giri Annamalai M, Jun 28 '19 at 08:08
@GiriAnnamalaiM You can refer https://stackoverflow.com/a/46635273 — giser_yugang, Jun 28 '19 at 08:25

score 2 · Accepted Answer · answered Jun 28 '19 at 05:33

formating the above prediction result

pred = ["2.8300792e-06","4.5637703e-03", "7.2316222e-02"," 6.7710824e-02"," 5.2243233e-01",
 "3.7763064e-04","1.2326813e-02","2.9277834e-01", "4.1662962e-03", "1.0876421e-05",
 "2.3830748e-06", "1.3590348e-04", "2.3074823e-02","3.3520879e-05", "4.0551484e-05",
 "1.9896568e-06" ,"1.0994432e-05", "4.7518920e-06" ,"2.3408763e-06" ,"6.7659844e-06"]


 pred_ = ["{:f}".format(float(x)) for x in pred])

 #np.argmax give you the position which have maximum value and not probability
 #o/p
 ['0.000003', '0.004564', '0.072316', '0.067711', '0.522432', '0.000378', '0.012327', 
 '0.292778', '0.004166', '0.000011', '0.000002', '0.000136', '0.023075', '0.000034', 
 '0.000041', '0.000002', '0.000011', '0.000005', '0.000002', '0.000007']



 np.argmax(pred_)
 #o/p
 4

score 1 · Answer 2 · answered Jun 28 '19 at 05:57

Try this code. from sklearn.metrics import classification_report import numpy as np

Y_test = np.argmax(y_test, axis=1) # Convert one-hot to index
y_pred = model.predict_classes(X_test)
print(classification_report(Y_test, y_pred))

Output of this code (Cifar 10)

     classes  precision    recall  f1-score   support

       0       0.82      0.40      0.54      1000
       1       0.84      0.66      0.74      1000
       2       0.47      0.51      0.49      1000
       3       0.41      0.50      0.45      1000
       4       0.44      0.72      0.55      1000
       5       0.56      0.43      0.49      1000
       6       0.69      0.71      0.70      1000
       7       0.80      0.52      0.63      1000
       8       0.62      0.85      0.72      1000
       9       0.73      0.73      0.73      1000

   micro avg       0.60      0.60      0.60     10000
   macro avg       0.64      0.60      0.60     10000
weighted avg       0.64      0.60      0.60     10000

Multi-class classification find probability of all classes

2 Answers2