Why is Tensorflow tf.image.decode_jpeg() rotating some images 90 degrees?

Question

I have many images which rotate 90 degrees when I use Tensorflow tf.image.decode_jpeg() to read in images in my data pipeline. When I look at those image on my computer they look correctly oriented and when I read the image with cv2 they also looks correctly oriented. However, when I read them in with Tensorflow tf.image.decode_jpeg(), they end up rotated 90 degrees. Anybody know why or how I can fix this so they read in the correct orientation (it doesn't happen to all images but many)?

To show what I mean, here's what a sample image looks like on my computer or when I read it with open-cv.

However, when I read it with Tensorflow (tensorflow 2.4.0) in my pipeline, many images rotate 90 degrees and, for example, the same image ends up looking like this.

To read and see an image with cv2 I do

import cv2
import matplotlib.pyplot as plt
import tensorflow as tf

%matplotlib inline

plt.imshow(cv2.imread(image_file_path)[:, :, ::-1]))

To read and see many rotated image with Tensorflow

def decode_images(dataset_file_paths):
    '''
    tf_read_converted_file_path = tf.io.read_file(dataset_file_paths) 
    return tf.image.decode_jpeg(tf_read_converted_file_path)

def check_processed_images(files_path_dataset, processing_function):
'''Plots images to see what the processing looks like when it is mapped in from a dataset. 
This can be used to, for example, look see how the augmentation of images look.

Arguments:
    files_path_dataset: TF TensorSliceDataset of filenames.
    processing_function: function which is used to map each element in the dataset.
    
Return:
    plot of images'''

augmentation_dataset = files_path_dataset.map(processing_function).shuffle(50).batch(16)

plt.figure(figsize=(13, 13))
for images, labels in augmentation_dataset.take(1):
    for i in range(16):
        ax = plt.subplot(4, 4, i + 1)
        plt.imshow(images[i].numpy())
        
####################
####################
# Look at a sample of images
# Take a directory of jpg images and create a TensorFlow dataset of filepaths
dirPath = type_in_a_directory_here
fileNames = tf.io.gfile.glob(f'{dirPath}/**/*.jpg')
fileNames = tf.random.shuffle(fileNames)
filenamesDs = tf.data.Dataset.from_tensor_slices(fileNames)

check_processed_images(filenamesDs, decode_images)

The code above will produce a grid of images so you can see. For example, all these images on my computer have the fishing lure so that they are horizontal but when I use TensorFlow to process my images many are rotated 90 degrees for some reason.

Am I doing something wrong that Tensorflow is rotating many of my images?

Camera orientation (potrait vs landscape) info in metadata could cause this. see https://stackoverflow.com/a/63798032/2580077 and https://stackoverflow.com/q/39132878/2580077 — duggi, Apr 09 '22 at 15:55

score 0 · Answer 1 · answered May 25 '21 at 08:15

SOLVED: I found out the problem. All the images are positioned correctly when viewing them as thumbnails and when opening them with the program "Preview" on my Mac. However, when I look at the actual dimensions of the images under their properties, many of them are actually supposed to look rotated 90 degrees. I have no clue why this is happening but as far as this, the problem has been discovered.

score -1 · Answer 2 · answered May 24 '21 at 14:30

It's not rotating the image, it's pyplot that represents the images with the format [W, H, D] (Widht, Height, Depth), while TensorFlow and OpenCV use the format [H,W,D].

If you want to visualize the image without rotation with pyplot, you should transpose it using tf.transpose

Why is Tensorflow tf.image.decode_jpeg() rotating some images 90 degrees?

2 Answers2