I want to standardise my image data for preprocessing for deep learning

Question

I am storing my images in gray_img_here which is a python list of 2d arrays. I want to standardise each image by applying: X = (X-u)/S where X is the pixel value, U is the mean of the image and S is the deviation of that pixel

def normalizeImages(self, gray_img_here):
    print "Normalizing the gray images..."
    print
    gray_img_numpy = np.array(gray_img_here)
    for i in range(len(gray_img_here)):
        print
        # print "mean of the {}th image", np.mean(gray_img_numpy[i])
        # print "std dev. of the {}th image", np.std(gray_img_numpy[i])
        # print
        gray_img_here[i] = float(gray_img_here[i] - np.mean(gray_img_numpy[i])) / float(np.std(gray_img_numpy[i], axis=0))

    return gray_img_here

However I get the error : gray_img_here[i] = float(gray_img_here[i] - np.mean(gray_img_numpy[i])) / float(np.std(gray_img_numpy[i], axis=0))

TypeError: only size-1 arrays can be converted to Python scalars

gray_img_here looks like this

[array([[ 37,  39,  41, ..., 119, 113, 109],
   [ 38,  40,  41, ..., 119, 113, 109],
   [ 39,  41,  42, ..., 117, 112, 108],
   ...,
   [ 25,  25,  26, ..., 168, 180, 182],
   [ 25,  26,  26, ..., 179, 191, 189],
   [ 26,  26,  26, ..., 184, 196, 191]], dtype=uint8), array([[ 91,  97, 101, ...,  48,  49,  51],
   [ 89,  93,  98, ...,  44,  45,  45],
   [ 85,  88,  94, ...,  40,  41,  41],
   ...,
   [137,  90,  52, ...,  35,  36,  36],
   [163, 103,  68, ...,  35,  35,  35],
   [216, 148, 107, ...,  35,  35,  34]], dtype=uint8), array([[ 64,  75,  93, ...,  85,  83,  82],
   [ 83,  93,  98, ...,  85,  81,  80],
   [ 91,  98,  96, ...,  84,  80,  81],
   ...,

Possible duplicate of [How to normalize a NumPy array to within a certain range?](https://stackoverflow.com/questions/1735025/how-to-normalize-a-numpy-array-to-within-a-certain-range) — Ywapom, Feb 08 '19 at 18:19
I already read that. I want to standardise my data not normalise it. — Sharan Narasimhan, Feb 08 '19 at 18:20
Not the error itsself, but I would look into this line gray_img_here[i] = float(gray_img_here[i] - np.mean(gray_img_numpy[i])) / float(np.std(gray_img_numpy[i], axis=0). I don't think this is doing what you think its doing. You want to use the mean of each array in the array, not that ith element — Edeki Okoh, Feb 08 '19 at 18:49
after reading the docs for np.std and mean it specifies that it is for the whole array. I also mentions that std works on a 'flattened array'. Does this mean a 1d array? — Sharan Narasimhan, Feb 08 '19 at 18:49
yes in this example, I am finding the mean and std for each array, not for each element — Sharan Narasimhan, Feb 08 '19 at 18:50

score 0 · Answer 1 · answered Feb 08 '19 at 21:04

It's a lot easier to standardise a 3D matrix. Something like:

X=np.array(gray_img_here,dtype=float)
Xm=X.mean(axis=0,keepdims=True)
Xstd=X.std(axis=(1,2),keepdims=True)
X_standardised=list((X-Xm)/Xstd)

The first line makes a 3D float array, so you don't have to worry about casting at every step. Then the easy standardisation and changing it back to a list as you like.

NOTE: There is no need to keep it as a list, you can have a 3D numpy array and address it the same way

I want to standardise my image data for preprocessing for deep learning

1 Answers1