Validation accurary doesn't get better

Question

I'm doing a project on Neural network and was trying a python code using keras and tensorflow package. Currently, I'm experiencing a problem of not getting the validation accurary to going up at all. I have a training set of 9815 images and 200 test set images. I'm really stuck here please help.

Right now, the validation result is at exactly 0.5000 for almost all 100 epoch and not going up at all.

#Image Processing Stage
train_data = ImageDataGenerator(rescale = 1./255, shear_range = 0.2, zoom_range = 0.2, horizontal_flip = True)

test_data = ImageDataGenerator(rescale = 1./255)

training_set = train_data.flow_from_directory('dataset/train_data', target_size = (128, 128),  batch_size = 42,  class_mode = 'binary')

test_set = test_data.flow_from_directory('dataset/test_data', target_size = (128, 128), batch_size = 42, class_mode = 'binary')




# Starting Convolutional Neural Network
start_cnn = load_model('CNN.h5')
start_cnn.get_weights()
start_cnn = Sequential()


start_cnn.add(Conv2D(32, (3, 3), input_shape = (128, 128, 3), activation = 'relu', padding='same'))                 #3*3*3*32+32
start_cnn.add(Conv2D(32, (3, 3), activation = 'relu'))
start_cnn.add(MaxPooling2D(pool_size = (2, 2)))

for i in range(0,2):
    start_cnn.add(Conv2D(128, (3, 3), activation = 'relu', padding='same'))

start_cnn.add(MaxPooling2D(pool_size = (2, 2)))

for i in range(0,2):
    start_cnn.add(Conv2D(128, (3, 3), activation = 'relu', padding='same'))

start_cnn.add(MaxPooling2D(pool_size = (2, 2)))

# Flattening
start_cnn.add(Flatten())

# Step 4 - Full connection
start_cnn.add(Dense(activation="relu", units=128))
start_cnn.add(Dense(activation="relu", units=64))
start_cnn.add(Dense(activation="relu", units=32))
start_cnn.add(Dense(activation="softmax", units=1))

start_cnn.summary()


# Compiling the CNN

start_cnn.compile(Adam(learning_rate=0.001), loss = 'binary_crossentropy', metrics = ['accuracy'])

start_cnn.fit(training_set, steps_per_epoch=234, epochs = 100, validation_data = test_set)  


start_cnn.save('CNN.h5')

Does this answer your question? [How to increase validation accuracy with deep neural net?](https://stackoverflow.com/questions/37020754/how-to-increase-validation-accuracy-with-deep-neural-net) — paisanco, Feb 22 '20 at 18:31

score 1 · Accepted Answer · answered Feb 22 '20 at 18:45

1

You cannot use the softmax activation with one neuron, like you are doing here:

start_cnn.add(Dense(activation="softmax", units=1))

To do binary classification with one neuron, you have to use the sigmoid activation:

start_cnn.add(Dense(activation="sigmoid", units=1))

answered Feb 22 '20 at 18:45

Dr. Snoopy

55,122
7
121
140

Thanks I'll try changing that. Just for clarification, the result from my CNN has 2 output choice, so for my last dense layer should i put unit of 1 or 2? – Win Rummaneethorn Feb 22 '20 at 18:52
@WinRummaneethorn The answer does not say to change the number of neurons, just the activation. – Dr. Snoopy Feb 22 '20 at 18:53
Sorry I just start out doing neural network, can you explain why can't softmax use for unit 1? – Win Rummaneethorn Feb 22 '20 at 19:24
@WinRummaneethorn Sure, because softmax is normalized with the sum of exponentials across all units, so with one unit the only possible value of this is a constant 1.0 You probably noticed that neither the training or validation loss/accuracy changes, and this is why. – Dr. Snoopy Feb 22 '20 at 19:33
Currently, after changing to sigmoid like you mentioned, my val_loss drastically decrease from about 8-10 to 1 now. Still, my val_accuracy is still at exact 0.5000 – Win Rummaneethorn Feb 22 '20 at 19:49

Validation accurary doesn't get better

1 Answers1