Loss is in negative. Accuracy is very low. I am unable to determine if my code is right or wrong

Question

I am trying to make a model that will detect numbers from an object. I have the images of the numbers at imgs and the code is in this file

I have 72 image each of the numbers: 0 , 1 , 4 , 5 , 6 , 7 , 8 , 9 (I don't need 2 and 3) in the imgs folder as mentioned above and using those as samples I am trying to train this model.

Image data set is at: imgs

code is done in jupyter notebook. Tensorflow 2.3.1

import numpy as np
import matplotlib.pyplot as plt
import os
import cv2

import tensorflow as tf
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D
import pickle

DATADIR = "E:/num/imgs"
CATEGORIES = ["0","1","4","5","6","7","8","9"]

for category in CATEGORIES:
    path = os.path.join(DATADIR, category)
    for img in os.listdir(path):
        img_array=cv2.imread(os.path.join(path,img), cv2.IMREAD_GRAYSCALE)
        plt.imshow(img_array, cmap="gray")
        plt.show()
        break
    break

print(img_array.shape)
IMG_SIZE = 100

new_array = cv2.resize(img_array, (IMG_SIZE, IMG_SIZE))
plt.imshow(new_array, cmap = "gray")
plt.show()

training_data = []

def create_training_data():
    for category in CATEGORIES:
        path = os.path.join(DATADIR, category)
        class_num = CATEGORIES.index(category)
        #print(class_num, " + ",category)
        #print(type(class_num), " + ",type(category))
        for img in os.listdir(path):
            try:
                img_array=cv2.imread(os.path.join(path,img), cv2.IMREAD_GRAYSCALE)
                new_array = cv2.resize(img_array, (IMG_SIZE, IMG_SIZE))
                #print(new_array, class_num)
                training_data.append([new_array, class_num])
            except Exception as e:
                pass
        
create_training_data()      


import random

random.shuffle(training_data)

x = []
y = []

for features, label in training_data:
    x.append(features)
    y.append(label)

x = np.array(x).reshape(-1, IMG_SIZE, IMG_SIZE, 1)
y = np.array(y)
print(type(y))

pickle_out = open("x.pickle", "wb")
pickle.dump(x, pickle_out)
pickle_out.close()

pickle_out = open("y.pickle", "wb")
pickle.dump(y, pickle_out)
pickle_out.close()


pickle_in = open("x.pickle", "rb")
x = pickle.load(pickle_in)

pickle_in = open("y.pickle","rb")
y = pickle.load(pickle_in)

x = x/255.0

model = Sequential()

model.add(Conv2D(256, (3, 3), input_shape=x.shape[1:]))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(256, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors

model.add(Dense(64))

model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

model.fit(x, y, batch_size=8, epochs=3, validation_split=0.3)

I am just starting to learn the whole thing and the results are as follows:

Epoch 1/3
51/51 [==============================] - 51s 998ms/step - loss: -1252177.3750 - accuracy: 0.1092 - val_loss: -7988151.0000 - val_accuracy: 0.1561
Epoch 2/3
51/51 [==============================] - 49s 969ms/step - loss: -61224024.0000 - accuracy: 0.1117 - val_loss: -200329888.0000 - val_accuracy: 0.1561
Epoch 3/3
51/51 [==============================] - 51s 992ms/step - loss: -623616768.0000 - accuracy: 0.1117 - val_loss: -1379303680.0000 - val_accuracy: 0.1561

<tensorflow.python.keras.callbacks.History at 0x1fd82f9bc70>

I see that the loss is in Negative, whereas I have seen many tutorials and no one has losses negative and my accuracy is also very low.

I have tried changing the batch_size, epochs and validation_split but no difference in result.

Did you normalize the input images? – akshayk07 Dec 12 '20 at 07:30 — akshayk07, Dec 12 '20 at 07:30

score 1 · Answer 1 · answered Dec 12 '20 at 07:40

This is because you are using sigmoid with binary cross-entropy. Binary cross-entropy is for losses which are like y_pred = [[0.6, 0.4], [0.4, 0.6]] and not a single value with sigmoid. Convert your output to the necessary format with 2 outputs instead of 1. Using one_hot is a good idea here as I think your output isn't binary but numbers. https://www.tensorflow.org/api_docs/python/tf/one_hot

score 1 · Answer 2 · answered Dec 12 '20 at 07:45

As this is a multiclass classification problem you should use a Softmax activation function for the classification head with the corresponding number of classes that you are trying to predict. The loss function for this should be categorical_crossentropy instead of binary_crossentropy as well.

model.add(Dense(len(CATEGORIES), activation="softmax"))

model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

score 1 · Answer 3 · answered Feb 02 '21 at 19:56

Activation

As per other answers, use softmax instead of sigmoid for multi-class single-label classifier. This gives us good weighted approximation across the output classes.
I assume, if you are using multi-class multi-label classifer, then sigmoid should would perform better and you want independently weighted prediction for each class.

Cost/Loss function

Irrespective of the choice of activation, please use categorical_crossentropy as you have multi-class.

Reference SO

Loss is in negative. Accuracy is very low. I am unable to determine if my code is right or wrong

3 Answers3

Activation

Cost/Loss function