0

I am working with Keras' sample denoising autoencoder; https://keras.io/examples/mnist_denoising_autoencoder/

As I compile it, I use the following options:

autoencoder.compile(loss='mse', optimizer= Adadelta, metrics=['accuracy'])

Followed by training. I did training deliberately WITHOUT using noisy training data(x_train_noisy), but merely tried to recover x_train.

autoencoder.fit(x_train, x_train, epochs=30, batch_size=128)

After training 60,000 inputs of MNIST digits, it gives me an accuracy of 81.25%. Does it mean there are 60000*81.25% images are PERFECTLY recovered (equaling to the original input pixel by pixel), that is, 81.25% output images from the autoencoder are IDENTICAL to their input counterparts, or something else?

Furthermore, I also conducted a manual check by comparing output and the original data (60000 28X28 matrices) pixel by pixel--counting non-zeros elements from their differences:

    x_decoded = autoencoder.predict(x_train)
    temp = x_train*255
    x_train_uint8 = temp.astype('uint8')
    temp = x_decoded*255
    x_decoded_uint8 = temp.astype('uint8')
    c = np.count_nonzero(x_train_uint8 - x_decoded_uint8)
    cp = 1-c /60000/28/28

Yet cp is only about 71%. Could any tell me why there is a difference?

Theron
  • 567
  • 1
  • 7
  • 21

1 Answers1

0

Accuracy doesn't make sense for a regression problem, hence the keras sample doesn't use that metric during autoencoder.compile.

In this case, keras calculates the accuracy as per this metric.

binary_accuracy

def binary_accuracy(y_true, y_pred):
    return K.mean(K.equal(y_true, K.round(y_pred)), axis=-1)

Using this numpy implementation, you should get the same value as output by Keras for validation accuracy at the end of training.

x_decoded = autoencoder.predict(x_test_noisy)
acc = np.mean(np.equal(x_test, np.round(x_decoded)))
print(acc)

Refer this answer for more details: What function defines accuracy in Keras when the loss is mean squared error (MSE)?

Manoj Mohan
  • 5,654
  • 1
  • 17
  • 21
  • I see, cheers a lot. So in my case, that is, I want to compare x_decocded and x_train pixel by pixel, shall I write my own accuracy function, or is there anything available in Keras I can adopt? Furthermore, by the nature of autoencoder, i.e, it tends to reproduce its input, is there any general guidelines about using choosing loss function and metrics? – Theron Jun 10 '19 at 07:37