Why does tensorflow decode jpeg images differently from scipy imread?

Question

tf.image.decode_jpeg() function of Tensorflow gives different numerical result than scipy.misc.imread() for jpg images. While the images look similar, pixel values are different.

import numpy as np
import scipy
import tensorflow as tf
import matplotlib.pyplot as plt
def minimal_example():
    def _bytes_feature(value):
        return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

    tffilename = 'astronaut.tfrecords'
    writer = tf.python_io.TFRecordWriter(tffilename)
    #image_source = 'https://upload.wikimedia.org/wikipedia/commons/8/88/Astronaut-EVA.jpg'
    image_path = 'astronaut.jpg'
    image_file = open(image_path,'rb')
    image = image_file.read()
    image_scipy = scipy.misc.imread(image_path)
    example = tf.train.Example(features=tf.train.Features(feature={'image':_bytes_feature(image)}))
    writer.write(example.SerializeToString())
    writer.close()

    record_iterator = tf.python_io.tf_record_iterator(path=tffilename)
    example = tf.train.Example()
    example.ParseFromString(next(record_iterator))
    image = example.features.feature['image'].bytes_list.value[0]
    image_tf = tf.image.decode_jpeg(image).eval(session=tf.Session())
    fig = plt.figure()
    ax1 = fig.add_subplot(121)
    ax2 = fig.add_subplot(122)
    ax1.imshow(image_scipy)
    ax2.imshow(image_tf)
    print('Reconstruction Error', np.sum(np.abs(image_tf - image_scipy)))
    plt.show()

result:

Reconstruction Error 3420883624

Is this a bug or am I doing something wrong?

score 3 · Accepted Answer · edited Jun 20 '20 at 09:12

The discrepancy arises because of inaccurate, but fast, default Discrete Cosine Tranform used by Tensorflow

According to the Source code

// The TensorFlow-chosen default for jpeg decoding is IFAST, sacrificing

// image quality for speed.

flags_.dct_method = JDCT_IFAST;

In order to get accurate decoding one can set the attribute dct_method = 'INTEGER_ACCURATE' as show in example below

def minimal_example():
    #image_source = 'https://upload.wikimedia.org/wikipedia/commons/8/88/Astronaut-EVA.jpg'
    image_path = 'astronaut.jpg'
    image_file = open(image_path,'rb')
    image_raw = image_file.read()
    image_scipy = scipy.misc.imread(image_path)
    image_tf = tf.image.decode_jpeg(image_raw).eval(session=tf.Session())
    image_tf_accurate = tf.image.decode_jpeg(image_raw,dct_method="INTEGER_ACCURATE").eval(session=tf.Session())
    print('Error For Default: ', np.sum(np.abs(image_tf - image_scipy)))
    print('Error For Accurate: ', np.sum(np.abs(image_tf_accurate - image_scipy)))
    #Error For Default:  3420883624
    #Error For Accurate:  0

score 2 · Answer 2 · answered Jul 19 '17 at 17:38

2

The JPEG standard does not require bit-to-bit identical decoding. So, some variations are expected between different implementations.

However, it still requires

a maximum 1 bit of difference for each pixel component.

So both outputs should not be apart further than one. Right?

print('max diff: ', np.max(np.abs(image_tf.astype(float) - image_scipy.astype(float))))
# max diff:  17.0

Ouch, at least one implementation does not follow the standard...

answered Jul 19 '17 at 17:38

P-Gn

23,115
9
87
104

First time i hear that. Very interesting! scipy seems to use libjpeg-dev (through PIL/Pillow), while tensorflow might use [libjpeg-turbo](https://github.com/tensorflow/tensorflow/issues/4807). – sascha Jul 19 '17 at 21:27

Why does tensorflow decode jpeg images differently from scipy imread?

2 Answers2

Linked