1

I am following a 2017 tutorial on deep learning in Python, and I am trying to standardize some images from the well-known MNIST dataset. The author has the following to say about the piece of code below (Decided to include it all, so anyone who might want to help can simply copy/paste and reproduce the issue).

It is also possible to standardize pixel values across the entire dataset. This is called feature standardization and mirrors the type of standardization often performed for each column in a tabular dataset. This is different to sample standardization described in the previous section as pixel values are standardized across all samples (all images in the dataset). In this case each image is considered a feature. You can perform feature standardization by setting the featurewise center and featurewise std normalization arguments on the ImageDataGenerator class.

Here is the code:

from keras.datasets import mnist
from keras.preprocessing.image import ImageDataGenerator
from matplotlib import pyplot as plt
from keras import backend as K
K.set_image_dim_ordering('th')

#Load data
(X_train, y_train), (X_test, y_test) = mnist.load_data()

# Reshape to be [samples][pixels][width][height]
X_train = X_train.reshape(X_train.shape[0], 1, 28, 28).astype('float32')
X_test = X_test.reshape(X_test.shape[0], 1, 28, 28).astype('float32')

datagen = ImageDataGenerator(featurewise_center=True, featurewise_std_normalization=True) #This standardizes pixel values across the entire dataset.

# Fit parameters from data
datagen.fit(X_train)

#Configure batch size and retrieve one batch of images
for X_batch, y_batch in datagen.flow(X_train, y_train, batch_size=9, shuffle=False):
    #Create grid of 3x3 images
    for i in range(0,9):
        plt.subplot(330 + 1 + i)
        plt.imshow(X_batch[i].reshape(28,28), cmap=plt.get_cmap('gray'))
    plt.show()
    break

I think that something has changed since 2017 in Keras especially in the ImageDataGenerator class, or in the flow function of that class, so that I now have to do this somewhat differently. The problem is: I can not find how. Any help would be appreciated

Current output: enter image description here

Expected output: enter image description here

Psychotechnopath
  • 2,471
  • 5
  • 26
  • 47

1 Answers1

2

ImageDataGenerator in Keras performs normalization across all images in your code example. You can check the actual X_batch values using the following code:

for X_batch, y_batch in datagen.flow(X_train, y_train, batch_size=9, shuffle=False):
    print(X_batch[0])
    break

The problem is related to the behavior of imshow, which is by default uses color normalization. Hence, imshow visualizes normalized images as non-normalized.

Andrey Sozykin
  • 926
  • 12
  • 13