I'm currently building a convolutional neural network to distinguish clear ECG images from ECG images with noise.
With Noise :
Without Noise :
My Problem
So I did build a convnet using keras above tensorflow and trained it several times but all the time, it has like 99% of Training Accuracy, 99% Validation Accuracy and 98% of Testing accuracy. But when I predict an image, it always give me [0].
Most of the times, my model early stops at epoch 3 or 4 with 99% of accuracy in both training and validation. It almost all the time given 98% or 99% accuracy in first epoch or second epoch.
My Model
from keras.models import Sequential
from keras.datasets import mnist
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation,Dropout,Flatten,Dense
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import TensorBoard
from keras.layers import ZeroPadding2D
from keras.optimizers import Adam
from keras.callbacks import EarlyStopping
tensorboard = TensorBoard(log_dir="./logs",histogram_freq=0,write_graph=True,write_images=True)
earlystop = EarlyStopping(monitor='val_loss',patience=2,verbose=1)
# Variables
batchSize = 15
num_of_samples = 15000
num_of_testing_samples = 3750
num_of_val_samples = 2000
training_imGenProp = ImageDataGenerator(rescale = 1./255,
width_shift_range=0.02,
height_shift_range=0.02,
horizontal_flip=False,
fill_mode='nearest'
)
testing_imGenProp = ImageDataGenerator(
rotation_range=5,
horizontal_flip=False,
fill_mode='nearest'
)
val_imGenProp = ImageDataGenerator(rescale = 1./255,
rotation_range=5,
zoom_range=0.2,
horizontal_flip=False,
fill_mode='nearest'
)
# Create the model
classifier = Sequential()
classifier.add(ZeroPadding2D(padding=(374,0),input_shape=(74,448,3)))
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Flatten())
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dropout(0.8))
classifier.add(Dense(units = 1, activation = 'sigmoid'))
classifier.summary()
adam = Adam(lr=0.00005)
classifier.compile(loss='binary_crossentropy',optimizer=adam,metrics=['accuracy'])
training_imGen = training_imGenProp.flow_from_directory(
'Directory\Training',
target_size=(74,448),
batch_size=batchSize,
class_mode='binary',
)
testing_imGen = testing_imGenProp.flow_from_directory(
'Directory\Testing',
target_size=(74,448),
batch_size=batchSize,
class_mode='binary',
)
val_imGen = testing_imGenProp.flow_from_directory(
'Directory\Validation',
target_size=(74,448),
batch_size=batchSize,
class_mode='binary',
)
classifier.fit_generator(
training_imGen,
callbacks = [tensorboard,earlystop],
steps_per_epoch=num_of_samples // batchSize,
epochs=30,
validation_data = val_imGen,
validation_steps = num_of_val_samples // batchSize
)
score, acc = classifier.evaluate_generator(
testing_imGen,
num_of_testing_samples // batchSize,
verbose = 0
)
print('Test score:', score)
print('Test accuracy:', acc)
classifier.save('Directory\Config_10_Model.h5')
Notes
I used 0.0005 Learning rate to stop this model being early stopped at 2nd or 3rd epoch. Also I've separated images for training, testing and validation under three folders and have 1020,375,200 images respectively for training,testing and validation (Which means Training folder alone has 2040 images since I have two classes. Each class have same number of images). So no images will be reused under any circumstances.
Also, before I'm rescaling images by 1./255 in ImageDataGenerator, My model had 50% of accuracy in training, validation and 54% in testing. But after using rescaling, this early stopping happened frequently and accuracy was 99% almost all the time.
I didn't use rescaling for test images purposely. But still receive 98% accuracy and yet fails desperately at predicting. Since I've with noise and without noise folders under training folder, My output class should be with noise or without noise. Since with Noise comes first in alphabetical order, I believe [0] class says With Noise and [1] should be for Without Noise. But if I input without noise image to model, it still gives me [0].
Below is the code I use to predict trained model.
from keras.models import load_model
import numpy as np
from keras.preprocessing import image
model = load_model('Directory\Config_10_Model.h5')
test_image = image.load_img('Path_to_Without_Noise_Image\image3452.png', target_size = (74, 448))
test_image = image.img_to_array(test_image)
test_image = test_image/255
test_image = np.expand_dims(test_image, axis = 0)
result = model.predict(test_image)
y_classes = result.argmax(axis=-1)
print(y_classes)
I don't know why this happenes even though I never used same images for testing, validation or training. Can someone help me with this? I tried everything and trained model with different hyper parameters but everytime this model output [0].
