0

I built this model and I am trying to predict two class labels crash and non-crash, I am not sure why I get multilabel indicator whereas it's a binary label, I saw a similar post but that is not helping me much. Could someone give some clue on what's wrong?

Initially, my code does not have this step predictions =np.argmax(predictions, axis=1) but I tried to see the stack answers for a similar questions but this does not work either.

Code:

from sklearn.preprocessing import LabelBinarizer

lb = LabelBinarizer()
labels = lb.fit_transform(y)

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42, test_size=0.2, stratify = y)

y_train = pd.get_dummies(y_train)
y_test = pd.get_dummies(y_test)

base_model = ResNet50(weights='imagenet', include_top=False)
X_train = base_model.predict(X_train)
X_train.shape
(2828, 7, 7, 2048)

# extracting features for validation frames
X_test = base_model.predict(X_test)
X_test.shape
(707, 7, 7, 2048)

X_train = X_train.reshape(2828, 7*7*2048)
X_test = X_test.reshape(707, 7*7*2048)
X_train.shape
(2828, 100352)

model = Sequential()
model.add(Dense(1024, activation='relu', input_shape=(100352,)))
model.add(Dropout(0.5))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(2, activation='softmax'))
from keras.callbacks import ModelCheckpoint
mcp_save = ModelCheckpoint('weight_resnet.hdf5', save_best_only=True, monitor='val_loss', mode='min')

#change loss function or optimizer and see

model.compile(loss='categorical_crossentropy',optimizer='Adam',metrics=['accuracy'])

model.fit(X_train, y_train, epochs=30, validation_data=(X_test, y_test), callbacks=[mcp_save], batch_size=128)

Epoch 1/30
23/23 [==============================] - 30s 1s/step - loss: 4.4351 - accuracy: 0.8248 - val_loss: 0.4182 - val_accuracy: 0.9194
Epoch 2/30
23/23 [==============================] - 33s 1s/step - loss: 2.1524 - accuracy: 0.8532 - val_loss: 0.2635 - val_accuracy: 0.9194
Epoch 3/30
23/23 [==============================] - 30s 1s/step - loss: 0.8864 - accuracy: 0.8610 - val_loss: 0.3656 - val_accuracy: 0.9194
Epoch 4/30
23/23 [==============================] - 30s 1s/step - loss: 0.4353 - accuracy: 0.8695 - val_loss: 0.4173 - val_accuracy: 0.9194
Epoch 5/30
23/23 [==============================] - 30s 1s/step - loss: 0.3068 - accuracy: 0.9098 - val_loss: 0.2600 - val_accuracy: 0.9194
Epoch 6/30
23/23 [==============================] - 29s 1s/step - loss: 0.2907 - accuracy: 0.9140 - val_loss: 0.2333 - val_accuracy: 0.9194
Epoch 7/30
23/23 [==============================] - 30s 1s/step - loss: 0.2235 - accuracy: 0.9272 - val_loss: 0.2326 - val_accuracy: 0.9194
Epoch 8/30
23/23 [==============================] - 30s 1s/step - loss: 0.2100 - accuracy: 0.9244 - val_loss: 0.2437 - val_accuracy: 0.9194
Epoch 9/30
23/23 [==============================] - 29s 1s/step - loss: 0.2364 - accuracy: 0.9223 - val_loss: 0.1988 - val_accuracy: 0.9194
Epoch 10/30
23/23 [==============================] - 29s 1s/step - loss: 0.2550 - accuracy: 0.9167 - val_loss: 0.1852 - val_accuracy: 0.9194
Epoch 11/30
23/23 [==============================] - 30s 1s/step - loss: 0.2310 - accuracy: 0.9129 - val_loss: 0.1717 - val_accuracy: 0.9194
Epoch 12/30
23/23 [==============================] - 29s 1s/step - loss: 0.2155 - accuracy: 0.9167 - val_loss: 0.1906 - val_accuracy: 0.9194
Epoch 13/30
23/23 [==============================] - 29s 1s/step - loss: 0.2220 - accuracy: 0.9173 - val_loss: 0.1890 - val_accuracy: 0.9194
Epoch 14/30
23/23 [==============================] - 30s 1s/step - loss: 0.1993 - accuracy: 0.9154 - val_loss: 0.1755 - val_accuracy: 0.9194
Epoch 15/30
23/23 [==============================] - 30s 1s/step - loss: 0.2066 - accuracy: 0.9120 - val_loss: 0.1591 - val_accuracy: 0.9194
Epoch 16/30
23/23 [==============================] - 29s 1s/step - loss: 0.2115 - accuracy: 0.9182 - val_loss: 0.1606 - val_accuracy: 0.9194
Epoch 17/30
23/23 [==============================] - 31s 1s/step - loss: 0.2015 - accuracy: 0.9163 - val_loss: 0.1618 - val_accuracy: 0.9194
Epoch 18/30
23/23 [==============================] - 29s 1s/step - loss: 0.1931 - accuracy: 0.9226 - val_loss: 0.1565 - val_accuracy: 0.9194
Epoch 19/30
23/23 [==============================] - 29s 1s/step - loss: 0.2232 - accuracy: 0.9181 - val_loss: 0.1571 - val_accuracy: 0.9194
Epoch 20/30
23/23 [==============================] - 31s 1s/step - loss: 0.2041 - accuracy: 0.9231 - val_loss: 0.1547 - val_accuracy: 0.9194
Epoch 21/30
23/23 [==============================] - 33s 1s/step - loss: 0.1996 - accuracy: 0.9171 - val_loss: 0.1583 - val_accuracy: 0.9194
Epoch 22/30
23/23 [==============================] - 30s 1s/step - loss: 0.2138 - accuracy: 0.9158 - val_loss: 0.2064 - val_accuracy: 0.9194
Epoch 23/30
23/23 [==============================] - 30s 1s/step - loss: 0.2230 - accuracy: 0.9215 - val_loss: 0.1706 - val_accuracy: 0.9194
Epoch 24/30
23/23 [==============================] - 29s 1s/step - loss: 0.2030 - accuracy: 0.9171 - val_loss: 0.1632 - val_accuracy: 0.9194
Epoch 25/30
23/23 [==============================] - 29s 1s/step - loss: 0.2236 - accuracy: 0.9125 - val_loss: 0.1806 - val_accuracy: 0.9194
Epoch 26/30
23/23 [==============================] - 29s 1s/step - loss: 0.2134 - accuracy: 0.9160 - val_loss: 0.1578 - val_accuracy: 0.9194
Epoch 27/30
23/23 [==============================] - 32s 1s/step - loss: 0.1839 - accuracy: 0.9195 - val_loss: 0.1345 - val_accuracy: 0.9194
Epoch 28/30
23/23 [==============================] - 29s 1s/step - loss: 0.1687 - accuracy: 0.9254 - val_loss: 0.1412 - val_accuracy: 0.9194
Epoch 29/30
23/23 [==============================] - 29s 1s/step - loss: 0.1687 - accuracy: 0.9215 - val_loss: 0.1371 - val_accuracy: 0.9194
Epoch 30/30
23/23 [==============================] - 31s 1s/step - loss: 0.2096 - accuracy: 0.9144 - val_loss: 0.1508 - val_accuracy: 0.9194
<tensorflow.python.keras.callbacks.History at 0x7f12ef5161d0>


from sklearn.metrics import classification_report
predictions = model.predict(X_test,batch_size = 32)
predictions =np.argmax(predictions, axis=1)


print(classification_report(y_test,predictions), target_names=lb.classes_)

Error:

ValueError                                Traceback (most recent call last)
<ipython-input-25-01c8b0565346> in <module>()
     17 
     18 
---> 19 print(classification_report(y_test,predictions), target_names=lb.classes_)
     20 # plot the training loss and accuracy
     21 N = 30

1 frames
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py in _check_targets(y_true, y_pred)
     88     if len(y_type) > 1:
     89         raise ValueError("Classification metrics can't handle a mix of {0} "
---> 90                          "and {1} targets".format(type_true, type_pred))
     91 
     92     # We can't have more than one value on y_type => The set is no more needed

ValueError: Classification metrics can't handle a mix of multilabel-indicator and binary targets

Here is a sample of my predictions and test data:

predictions

array([[3.1774469e-02, 9.6822548e-01],
       [2.2356339e-01, 7.7643663e-01],
       [4.0342213e-04, 9.9959654e-01],
       ...,
       [2.4084024e-01, 7.5915980e-01],
       [5.5107642e-02, 9.4489235e-01],
       [4.9178453e-07, 9.9999952e-01]], dtype=float32)

y_test

      crash  noncrash
395       0         1
114       0         1
2138      0         1
731       1         0
3436      0         1
...     ...       ...
404       0         1
216       0         1
3253      0         1
3322      0         1
1932      0         1

[707 rows x 2 columns]
desertnaut
  • 57,590
  • 26
  • 140
  • 166
KSp
  • 1,199
  • 1
  • 11
  • 29
  • Please post a sample of your `predictions` and `y_test`. And keep in mind that any code that comes *after* the error (like your plotting code here) is irrelevant to the issue (never executed) and it should not be included here as it just creates unnecessary clutter (edited out). – desertnaut Feb 10 '21 at 21:58
  • Please find the update on the variable predictions and y_test. Thanks for editing my code. – KSp Feb 10 '21 at 22:04
  • 1
    No problem. But these predictions imply you have not actually run `predictions =np.argmax(predictions, axis=1)`, although you have included it in your code; what is the case, exactly? – desertnaut Feb 10 '21 at 22:07
  • As the duplicate explains, *both* your `predictions` and `y_test` should be single digits in order for scikit-learn functions like `classification_report` and `confusion_matrix` to work, *not* one-hot encoded vectors. – desertnaut Feb 10 '21 at 22:10
  • I did run this part of the code predictions =np.argmax(predictions, axis=1) and that's why I got this error. So are you saying I should not have used label binarizer? – KSp Feb 10 '21 at 22:12
  • It is not needed here for binary classification, but using it is not wrong either; you just need to "undo" it back to single digits afterwards if you want to use the relevant scikit-learn functionality. – desertnaut Feb 10 '21 at 22:14
  • 1
    @desertnaut is right. The question is unable to receive an answer. So I added an answer to its marked question, [here](https://stackoverflow.com/a/66155191/9215780). – Innat Feb 11 '21 at 12:56

0 Answers0