2

Let's say I parse an image file with multiple different libraries, and ask the library for the RGB value at pixel (20, 30).

Under what conditions can I expect to get consistent results across libraries and library versions?

Intuitively, I would assume that with simpler formats like PPM or (with some constraints) BMP I could probably expect consistent results, and with JPEG I'd get results all over the place even in relatively simple cases with no way to avoid that.

That leaves me thinking about PNG: If I take an input image, convert it to a PNG with a defined color depth (e.g. 8-bit-per-channel RGBA, with all transparency values set to fully opaque) and no color profile, should I be able to expect:

  1. all common libraries to interpret the resulting PNG in the same way (yielding the same array of RGB(A) values when reading the file)?

  2. all common libraries to be able to turn said array of RGB(A) values back into a PNG that all common libraries will interpret in the same way?

(Obviously, the file bytes themselves will likely be different due to metadata, order of packets, etc. - I'm just talking about pixel values here. Also, obviously the initial conversion step may change the image if the original input had a color profile etc.)

For example, if you get this sample file:

wget https://upload.wikimedia.org/wikipedia/commons/thumb/7/7a/Sun_getting_through_fog_in_the_New_Zealand_bush%2C_Bryant_Range.jpg/500px-Sun_getting_through_fog_in_the_New_Zealand_bush%2C_Bryant_Range.jpg

then decode with Python:

import PIL.Image                                                                               
img = PIL.Image.open('500px-Sun_getting_through_fog_in_the_New_Zealand_bush,_Bryant_Range.jpg')
print(img.getpixel((100,100)))  # prints (73, 50, 60)

you will get different results than with Golang:

package main

import (
    "fmt"
    "image"
    "log"
    "os"

    "image/color"
    _ "image/jpeg"
    _ "image/png"
)

func main() {
    reader, err := os.Open("500px-Sun_getting_through_fog_in_the_New_Zealand_bush,_Bryant_Range.jpg")
    if err != nil {
        log.Fatal(err)
    }
    m, _, err := image.Decode(reader)
    if err != nil {
        log.Fatal(err)
    }
    c := m.At(100, 100).(color.YCbCr)
    fmt.Printf("%+v\n", c)
    r, g, b := color.YCbCrToRGB(c.Y, c.Cb, c.Cr)
    fmt.Printf("%v %v %v\n", r, g, b)  // prints 72 50 59
}

GIMP decodes pixel (100, 100) as (73, 50, 60), i.e. same as PIL, if you select "Keep" in the profile dialog.

Jan Schejbal
  • 4,000
  • 19
  • 40
  • There’s no reason why different libraries would give you different values for the same pixel in any file. A JPEG file uses lossy compression, but the loss happens when saving, once saved, any compliant reader will load the same data back in. – Cris Luengo Jul 10 '20 at 00:58
  • PNG is lossless. You can read and write them anywhere and expect no changes in pixel values. – Cris Luengo Jul 10 '20 at 00:59
  • See e.g. [this lengthy thread](https://groups.google.com/forum/#!topic/rec.photo.digital/yAxoW9HyHPQ) on the various issues that could cause differences in JPEG decoding. And while I understand that PNG is lossless, the same PNG will often display with different colors in different applications, e.g. based on [color profiles, gamma correction, etc.](http://www.libpng.org/pub/png/book/chapter10.html). As I mentioned, this is just my understanding so far, so I could be wrong of course. – Jan Schejbal Jul 10 '20 at 01:34
  • Added examples. – Jan Schejbal Jul 10 '20 at 02:32
  • 1
    The differences you see are likely caused by applying or not applying gamma and/or color profiles stored in the file. There is one way of decoding the JPEG data, and there is no reason that this cannot be done consistently. The linked thread discusses rounding errors (unlikely to cause important differences) and implementation errors. So yes, some software might implement the JPEG standard wrong and hence produce wrong pixel values. Can't help that. – Cris Luengo Jul 10 '20 at 04:33
  • It's good to know that in theory this shouldn't happen; given that it clearly does (in my case, I consider a difference of 1 important, as the goal is to get *exactly* the same values), I'm interested in the practical aspects of what I can reasonably rely on and what not. – Jan Schejbal Jul 21 '20 at 21:05

2 Answers2

2

Among the advantages of PNG is:

Losslessness: No loss: filtering and compression preserve all information.

Therefore, assuming filtering and compression were (un)done properly, the color values in a PNG are guaranteed consistent.

Importantly, transparency (alpha) is stored in a PNG non-premultiplied; so, ensure the equivalent of getpixel() in each language you use does not premultiply if you want the raw values. (If consistently cooked values will suffice, ensure the functions all premultiply.)

Note: The difference you observed in Golang probably was due to the conversion from RGB to YCbCr.

The authoritative source: https://www.w3.org/TR/PNG (contains 15 matches for "loss")

greg-tumolo
  • 698
  • 1
  • 7
  • 30
  • 1
    "[W]ith all transparency values set to fully opaque", premultiplication is unimportant. – greg-tumolo Jul 21 '20 at 23:17
  • Thank you! I understand that PNG is lossless, but I am particularly worried about color space conversions/corrections that libraries may automatically apply. For example, I assume images with `gAMA`, `cHRM`, `iCCP` or `sRGB` chunks could potentially be decoded differently based on the decoder, and the spec also says *"When the incoming image has unknown gamma (...), choose a likely default gamma value"*. – Jan Schejbal Jul 23 '20 at 03:33
  • Based on my testing, at least `convert`, Golang's default image library and and Python's PIL seem to return the raw values and ignore `gAMA`, so in practice I expect to be fine. Thanks for pointing me to the spec. Do you know if this specific behavior (non-displaying decoders not applying gamma correction etc.) is a) universal, b) specified/documented somewhere? I only found a warning not to change the values in the *encoder* section of the spec. – Jan Schejbal Jul 23 '20 at 03:38
  • I've added my research in a separate answer. I'll leave the bounty open for now; if someone adds an authoritative answer regarding how this is handled by libraries I'll award a bounty to both your answer and theirs. Thank you again! – Jan Schejbal Jul 23 '20 at 04:24
  • 1
    You are welcome (again)! a) I do not. b) I do: https://stackoverflow.com/a/63046854/13744178 ;) – greg-tumolo Jul 23 '20 at 10:49
2

JPEG

Due to the various conversions involved (JPEG encodes colors in YCbCr), decoded colors can differ, as demonstrated in the example code in the question (see also).

PNG

TL;DR: In practice, it seems like PNG is able to reliably preserve exact color values across languages/libraries.

PNG stores RGB (or greyscale) values (potentially using a palette). While it does support e.g. gamma correction with the gAMA chunk and various forms of color management (e.g. the cHRM, iCCP, sRGB chunks), libraries seem to often ignore them.

I created an image containing #808080 grey, added various gAMA values, and all of the following reported (128, 128, 128) as the color value for all files:

  • ImageMagick (convert file.png -crop 1x1+5+5 -depth 8 txt:-)
  • GIMP (when opening the image and using the info panel, all images also appear equally bright)
  • Python PIL/Pillow
  • Golang
  • Python ImageIO in default mode.

ImageIO is interesting because it has an ignoregamma option which defaults to True. Disabling it will result in the grey value being interpreted as 186 instead of 128 in an image saved from GIMP. This is a sufficiently extreme difference that established libraries are unlikely to suddenly change their default behavior:
comparison between grey-128 and grey-186

The Golang PNG reader source code confirms that does not seem to interpret the additional color information chunks.

It's probably a good idea to omit those chunks. Firefox interprets the PNG without a gAMA chunk as having color (128, 128, 128). The PNG with the original gAMA chunk is also correctly interpreted; a PNG where GIMP embedded an ICC profile is already off by 1, showing (127, 127, 127), and the PNGs with edited gamma values are very different (as expected).

Jan Schejbal
  • 4,000
  • 19
  • 40
  • 1
    For peace of mind, you should strip all noncritical chunks from the PNG. The [critical chunks](https://www.w3.org/TR/PNG/#4Concepts.FormatTypes) are IHDR, (PLTE, ) IDAT, and IEND. – greg-tumolo Jul 23 '20 at 11:51