ImageIO reading slightly different RGB values than other methods

Question

I've found that I'm getting different RGB when using Java (& actually paint.NET) than I am using ImageMagick, Gimp, Python, and Octave. The last 4 all agreeing with eachother and so I'm assuming to be correct.

For these examples, I'm using this test image: http://farm3.static.flickr.com/2811/9177301733_9836174725_o.jpg

Testing pixel x=4144 y=2768

               R    G    B
Java        = (125, 107, 69)
Paint.NET   = (125, 107, 69)
ImageMagick = (128, 106, 67)
Python      = (128, 106, 67)
Octave      = (128, 106, 67)
Gimp        = (128, 106, 67)

What gives?

Here's a quick test using imagemagick:

convert image.jpg -crop 1x1+4144+2768 -depth 8 txt:

output:

# ImageMagick pixel enumeration: 1,1,65535,srgb
0,0: (32896,27242,17219)  #806A43  srgb(128,106,67)

Here's some java and python code that also demonstrates the problem:

import org.apache.commons.io.FileUtils;
import org.junit.Test;

import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.net.URISyntaxException;
import java.net.URL;

public class ImageIOTest {
    @Test
    public void can_read_file() throws IOException, InterruptedException, URISyntaxException {
        File tempFile = File.createTempFile("image", "jpg");
        FileUtils.copyURLToFile(new URL("http://farm3.static.flickr.com/2811/9177301733_9836174725_o.jpg"), tempFile);

        BufferedImage image = ImageIO.read(tempFile);

        int javaRGB = image.getRGB(4144, 2768);
        int javaRed = (javaRGB >> 16) & 0xFF;
        int javaGreen = (javaRGB >> 8) & 0xFF;
        int javaBlue = (javaRGB >> 0) & 0xFF;
        System.out.printf("rgb: (%d, %d, %d)", javaRed, javaGreen, javaBlue);
    }
}

And here is the corresponding python script:

from PIL import Image
import sys, urllib, cStringIO

file = cStringIO.StringIO(urllib.urlopen("http://farm3.static.flickr.com/2811/9177301733_9836174725_o.jpg").read())

im = Image.open(file)
pix = im.load()
print pix[4144, 2768]

I've tried using this 12monkeys library in the hope that that would fix it but no dice. Any other ideas how I can extract correct RGB values using java? Surely I'm not the first person to have this problem!

Update

I tried getRaster().getSample() but got the same invalid result: System.out.println(raster.getSample(4144, 2768, 0)+","+ raster.getSample(4144, 2768, 1)+","+ raster.getSample(4144, 2768, 2)); output: 125,107,69

More info

Here is some output that shows what RGB values are decoded by three different tools for the first 9 (3x3 square of) pixels in the top left of the image. As you can see, Python and ImageMagick are in unison. Java sometimes matches. I've put an X where java disagrees...:

Tool          [x, y] = (R , G , B )
ImageIO     : [0, 0] = (86, 90, 93)
Python      : [0, 0] = (86, 90, 93)
ImageMagick : [0, 0] = (86, 90, 93)

ImageIO     : [1, 0] = (86, 90, 93)
Python      : [1, 0] = (86, 90, 93)
ImageMagick : [1, 0] = (86, 90, 93)

ImageIO     : [2, 0] = (90, 91, 95) X
Python      : [2, 0] = (88, 92, 95)
ImageMagick : [2, 0] = (88, 92, 95)

ImageIO     : [0, 1] = (85, 93, 95)
Python      : [0, 1] = (85, 93, 95)
ImageMagick : [0, 1] = (85, 93, 95)

ImageIO     : [1, 1] = (85, 93, 95) X
Python      : [1, 1] = (87, 92, 95)
ImageMagick : [1, 1] = (87, 92, 95)

ImageIO     : [2, 1] = (87, 92, 95)
Python      : [2, 1] = (87, 92, 95)
ImageMagick : [2, 1] = (87, 92, 95)

ImageIO     : [0, 2] = (83, 93, 94)
Python      : [0, 2] = (83, 93, 94)
ImageMagick : [0, 2] = (83, 93, 94)

ImageIO     : [1, 2] = (83, 93, 94) X
Python      : [1, 2] = (84, 92, 94)
ImageMagick : [1, 2] = (84, 92, 94)

ImageIO     : [2, 2] = (83, 91, 93)
Python      : [2, 2] = (83, 91, 93)
ImageMagick : [2, 2] = (83, 91, 93)

Why is Java giving different values for some pixels? Alternatively, is there another (fast) way to generate correct values using native Java code?

UPDATE 2016-09-26:

I committed my code that demonstrates this problem and pushed it to github (imageio-test) so that I could easily test it out on different machines. It turns out that Java was consistent across both OSX and Ubuntu Linux, but it was Python, ImageMagick and Octave that were inconsistent. In other words, on the Linux box, all tools agree with each other, and therefore, I'm now thinking that java was right all along, and it's the other tools that are giving incorrect results on OSX! I still don't really understand why and I haven't got any concrete proof as to which values are the correct ones but I'm getting somewhere...

Answer 1

Actually, I'd like turn the problem around, and say I'm surprised that so many different platforms and tools actually produce the same values. :-)

JPEG is lossy

First of all, JPEG is a lossy image compression method. This means that reproducing the exact data of the original is not possible. Or, if you like, several different pixel values may all be "correct" in some way.

The technical reasons why not all JPEG software produce the exact same values from the same source file, is typically different rounding/clamping of values, or integer approximations of floating point operations for better performance. Other variations may stem from different interpolation algorithms applied to restore the subsampled chroma values, for example (ie. a smoother image may look more pleasing to the eye, but isn't necessarily more correct).

Another excellent answer to a similar question states that "The JPEG standard does not require that decoder implementations produce bit-for-bit identical output images", and quotes the Wikipedia JPEG entry:

[...] precision requirements for the decoding [...]; the output from the reference algorithm must not exceed:

• a maximum of one bit of difference for each pixel component
• low mean square error over each 8×8-pixel block
• very low mean error over each 8×8-pixel block
• very low mean square error over the whole image
• extremely low mean error over the whole image

(Note that the above talks about the reference implementation only).

However, with some luck, it seems that all of your software/tools actually end up using (some version of) libjpeg. Because they all use libjpeg, the source of the differences you see is most likely not related to the JPEG decoding.

Color Spaces

Even if all your software converts the JPEG file to a representation using RGB values, there could be differences in the color space they use for this representation.

It does seem that all of the software you are using actually displays the RGB values in the sRGB color space. This is probably the most standard and widely used color space used in mainstream computing, so that is no surprise after all. As the color space is always sRGB, the source of the differences you see is most likely not the color space.

ICC profiles and color matching

The next possible source of color differences, is that color matching (as done by a Color Matching Module, CMM or Color Management System, CMS) is not an 100% exact science (see for example this document on black point compensation or read some of the more technical posts from the Little CMS blog).

Most likely the software running on Mac OS X are using Apple's CMM, while Java is using Little CMS always (from OpenJDK 7 or Oracle JDK/JRE 8), and most software on the Linux platform will likely also use the open source Little CMS (according to the Little CMS home page, "You can find Little CMS in most Linux distributions"). Software on Windows will likely deviate slightly as well (I haven't been able to verify if Paint.Net uses Little CMS, Windows' built in CMM or something else). And of course, using Adobe's CMM (ie. Photoshop) will likely deviate as well.

Again, with some luck, a lot of the software you tested uses the same CMM or CMS engine, Little CMS, so again you will have a lot of equal results. But it seems that some of the software you tested uses different CMMs, and is a probable source of the slight color differences.

In summary

The different pixel values you see are all "correct". The differences stem from different implementations or approximations of algorithms in software, but that does not necessarily mean that one value is correct and the others are wrong.

PS: If you need to reproduce the exact same values across multiple platforms, use the same tool stack/same algorithms on all platforms.

Answer 2

As per my comment, the major difference between the various applications/libraries that you've used to retrieve pixel colour value is that they're all using different versions of libjpeg - at least on Mac OSX.

When you check out your Github project onto certain versions of Ubuntu you'll see that all values are reported the same across the board. In these cases, python ImageMagick and the Java JDK/JRE are using the same libjpeg implementation.

On the Mac, if you installed jpeg via homebrew, or Pillow via pip then you'll notice that they're using libjpeg v9 (libjpeg.9.dylib), whereas Java 7 and 8 JDKs come with their own libjpeg bundled that are quite different.

Octave lists its jpeg dependencies as libjpeg8-dev.

GIMP, Inkscape, Scribus etc also come bundled with their own. In my case, GIMP comes bundled with the same version as python and ImageMagick, which would explain the similar values (ie: /Applications/GIMP.app/Contents/Resources/lib/libjpeg.9.dylib)

If you want to guarantee the values being the same across apps, you have options:

1. Stick to the same platform/stack (as suggested by @haraldk) - Stick with developing/running your stuff on Linux platforms that guarantee all of them use the same libjpeg version
2. Bind your Java code to the same version that the other apps are using - ie: load libjpeg.9.dylib and use that from your Java app. I'm not 100% sure how you'd do that though.
3. Recompile your JDK to use the right one - an option referenced by this answer is to use openjdk and compile it against the desired version of libjpeg, which sounds more achievable.

I'll admit that options 2 and 3 are really harder versions of option 1!

Note:
I'm definitely up-voting @haraldk's answer because his conclusion is pretty much the same.

I also played with using different icc profiles, and they give totally different answers. So it's worth being wary of that.

I just wanted to add an answer that added more emphasis on the libjpeg implementation, because I believe that is what is catching you out in your specific instance.

Update

Actually, there is another major difference noted in @haraldk's answer, being the diff between CMM and Little CMS. As he says: Java uses Little CMS, which is also used by Ubuntu

I actually think that's more likely to be the answer here.

Answer 3

Color consistency and ICC Profiles

Java doesn't respect the color profile when uploading an image. Also different OS are differently handle RGB colors.

Here is what Oracle writes about import java.awt.color:

Typically, a Color or ColorModel would be associated with an ICC Profile which is either an input, display, or output profile. There are other types of ICC Profiles, e.g. abstract profiles, device link profiles, and named color profiles, which do not contain information appropriate for representing the color space of a color, image, or device. Attempting to create an ICC_ColorSpace object from an inappropriate ICC Profile is an error.

ICC Profiles represent transformations from the color space of the profile (e.g. a monitor) to a Profile Connection Space (PCS). Profiles of interest for tagging images or colors have a PCS which is one of the device independent spaces (one CIEXYZ space and two CIELab spaces) defined in the ICC Profile Format Specification. Most profiles of interest either have invertible transformations or explicitly specify transformations going both directions. Should an ICC_ColorSpace object be used in a way requiring a conversion from PCS to the profile's native space and there is inadequate data to correctly perform the conversion, the ICC_ColorSpace object will produce output in the specified type of color space (e.g. TYPE_RGB, TYPE_CMYK, etc.), but the specific color values of the output data will be undefined.

The details of ICC_ColorSpace class are not important for simple applets, which draw in a default color space or manipulate and display imported images with a known color space. At most, such applets would need to get one of the default color spaces via ColorSpace.getInstance(). (excerpt from docs.oracle.com) https://docs.oracle.com/javase/7/docs/api/java/awt/color/ICC_ColorSpace.html

Colorspace Transformations in Java

Colorspace transformations are controlled by the destination type for both reading and writing of images. When Rasters are read, no colorspace transformation is performed, and any destination type is ignored. A warning is sent to any listeners if a destination type is specified in this case. When Rasters are written, any destination type is used to interpret the bands. This might result in a JFIF or Adobe header being written, or different component ids being written to the frame and scan headers. If values present in a metadata object do not match the destination type, the destination type is used and a warning is sent to any listeners. (excerpt from docs.oracle.com) https://docs.oracle.com/javase/7/docs/api/javax/imageio/metadata/doc-files/jpeg_metadata.html

Useful links

Look at info touching RGB conversion. There are some issues on normalized float/int color components by Rolf W. Rasmussen:

http://opensource.apple.com/source/gcc3/gcc3-1041/libjava/java/awt/image/ColorModel.java

Read The Sad Story of PNG Gamma “Correction” (The problem is: JPEG and TIFF suffer from the same "disease").

https://hsivonen.fi/png-gamma/

Look at S.O. post. There's possible solution for you:

In Java converting an image to sRGB makes the image too bright

If you still have inconsistent colors after all attemps, try to convert images to sRGB profile, but do not embed them:

https://imageoptim.com/color-profiles.html

Also, I hope the book by Kenny Hunt could be useful.

...and the following code (published at www.physicsforums.com) allows you to see how various RGB's look like:

import java.awt.*; 
import javax.swing.*; 

public class RGB { 
    public static void main(String[] args) { 
        JFrame frame = new JFrame("RGB"); 
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE); 
        RGBpanel panel = new RGBpanel(); 
        frame.getContentPane().add(panel); 
        frame.pack(); 
        frame.setVisible(true); 
    } 
} 

class RGBpanel extends JPanel { 
    public RGBpanel() { 
        setPreferredSize(new Dimension(300,300)); 
        int red = Integer.parseInt(JOptionPane.showInputDialog("Enter red value")); 
        int green = Integer.parseInt(JOptionPane.showInputDialog("Enter green value")); 
        int blue = Integer.parseInt(JOptionPane.showInputDialog("Enter blue value")); 
        Color colour = new Color(red,green,blue); 
        setBackground(colour); 
    } 
}

Recap

The problem of color inconsistency stems from color profiles. Try to assign uniform color profile to all images whether manually (in Photoshop) or programmatically (in Java).

Answer 4

See here: WARNING: Color space tagged as sRGB, without an embedded color profile. Windows and Mac browsers and apps treat the colors randomly.

Edit: As far as rounding errors and implementation variance by version; they're simply not the case for this image. There is some magic going on with the Mac that makes blue and green brighter on a color matching curve. Correct the color space, the color matching will give the same result. I up-voted Andy Fedoroff's answer, but I also notice no one has actually given you a solution... You've come to the conclusion that Java is correct. Go with that. Libjpeg hasn't changed in a long time. It's stable and reproduces colors reliably across many platforms and environments. Significant (in any way) changes have not been made to decoding the aged standard jpeg.

Edit 2: Going to try to create an example that produces the same values as the Mac profile based on your project. Need your Mac's factory ICC profile from Library/ColorSync/Profiles.

Here's where I'm at. Here's an example with the sRGB ICC v4 profile applied. This is technically applying sRGB over sRGB, but it's explaining the concept.

private ICC_Profile cp = ICC_Profile.getInstance("src/test/resources/sRGB_ICC_v4_Appearance.icc");
private ICC_ColorSpace cs = new ICC_ColorSpace(cp);

private int[] getRGBUsingImageIO2(File file, int x, int y) throws IOException {
    BufferedImage image = ImageIO.read(file);
    ColorConvertOp cco = new ColorConvertOp( cs, null );
    BufferedImage result = cco.filter( image, null );
    int javaRGB = result.getRGB(x, y);
    int javaRed = (javaRGB >> 16) & 0xFF;
    int javaGreen = (javaRGB >> 8) & 0xFF;
    int javaBlue = (javaRGB >> 0) & 0xFF;

    return new int[]{javaRed, javaGreen, javaBlue};
}

Image IO 1  : [0, 0] = [145, 146, 164] 
Image IO 2  : [0, 0] = [145, 147, 165] 
Image IO 1  : [1, 0] = [137, 138, 156] 
Image IO 2  : [1, 0] = [137, 139, 157] 
Image IO 1  : [2, 0] = [148, 147, 161] 
Image IO 2  : [2, 0] = [148, 148, 162] 
Image IO 1  : [0, 1] = [150, 153, 168] 
Image IO 2  : [0, 1] = [150, 154, 169] 
Image IO 1  : [1, 1] = [138, 141, 156] 
Image IO 2  : [1, 1] = [138, 142, 157] 
Image IO 1  : [2, 1] = [145, 147, 159] 
Image IO 2  : [2, 1] = [145, 148, 160] 
Image IO 1  : [0, 2] = [154, 160, 172] 
Image IO 2  : [0, 2] = [154, 161, 173] 
Image IO 1  : [1, 2] = [146, 152, 164] 
Image IO 2  : [1, 2] = [146, 153, 165] 
Image IO 1  : [2, 2] = [144, 148, 157] 
Image IO 2  : [2, 2] = [144, 149, 158] 

Could you commit your color profile to your imageio-test repo?