Static image shown when saving PIL images to 4D Numpy array

Question

I have the following Python code where I am saving some images of shape 326x490x3 as numpy arrays for pre-processing at a later stage. I want to save my images in a 4D numpy array so that I can process them in batches later. The code works fine, but I found out that when I convert each 3D element of the 4D array back to an RGB image, I just get a static image.

CODE:

data = np.zeros((129, 326, 490, 3))
image_path = '0.jpg'
img = Image.open(image_path)
data[0,:,:,:] = np.asarray(img)
im = Image.fromarray(data[0], 'RGB')
im.show()

OUTPUT:

But when I try to display the 3D numpy array slice from the 4D array as a grayscale image it works fine.

CODE:

data = np.zeros((129, 326, 490, 3))
image_path = '0.jpg'
img = Image.open(image_path)
data[0,:,:,:] = np.asarray(img)
im = Image.fromarray(np.dot(data[0], [0.299, 0.587, 0.114]))
im.show()

OUTPUT:

The solution given here works as expected when I save the image to a 3D numpy array and switch back to a PIL image.

CODE:

data = np.zeros((129, 326, 490, 3))
image_path = '0.jpg'
img = Image.open(image_path)
im = Image.fromarray(np.asarray(img), 'RGB')
im.show()

OUTPUT:

Can someone please explain this behavior? I don't understand how the code works as expected for a 3D numpy array, but works differently for a 3D array slice of a 4D numpy array.

Answer 1

The default data type of the array created by numpy.zeros is numpy.float64 (i.e. floating point). So data is a floating point array. In the line im = Image.fromarray(data[0], 'RGB'), you have explicitly specified the mode to be 'RGB', which means 8 bit integers (see the Modes documentation), so fromarray interprets the argument data[0] as an array of 8 bit integers. Apparently it doesn't try to convert the input array; it simply assumes that the underlying data in the array is stored as 8 bit integers. Since data[0] actually contains floating point values, the result is not correct.

In the case where you use im = Image.fromarray(np.dot(data[0], [0.299, 0.587, 0.114])), you haven't explicitly specified the mode, so fromarray uses its own code to determine the mode, which in this case will be 'F' (32 bit floating point). So it correctly converts your data. If you had, for example, specified the mode to be 'L' (meaning 8 bit black and white) (i.e. im = Image.fromarray(np.dot(data[0], [0.299, 0.587, 0.114]), 'L')), the call would succeed, but the image data would again be incorrect, because fromarray would be interpreting memory that contains floating point values as if it contained 8 bit integer pixels.

Probably the simplest fix is to create data as an array of 8 bit unsigned integers:

data = np.zeros((129, 326, 490, 3), dtype=np.uint8)