Why does Pillow convert return colours outside the specified palette?

Question

Using Pillow 5.4.1, Python 3.6.8

Given an image image.png with 9 distinct colours, and given a data palette with 5 distinct colours, one would expect that asking pillow to reduce the image to the described palette that the resulting image would contain colours from only that palette.

However, using the im.im.convert method returns an image with colours outside the specified palette; specifically they are always greyscale images (R==B==G values)

Sample Code, outputting the unique set of colours for the original image, palette, and converted image.

from PIL import Image
im = Image.open("image.png")

# create palette from raw data
# colours: Red, Green, Blue, Black, and White (5 total)
RGBBW = [(255,0,0), (0,255,0), (0,0,255), (0,0,0), (255,255,255)]
data = sum([list(x) for x in RGBBW], [])[:256]
pimg = Image.new("P",(16,16))
pimg.putpalette(data)

# Hack
im.convert("RGB")
cim_ = im.im.convert("P", 0, pimg.im)
cim = im._new(cim_).convert("RGB")

def colors(im):
    cs = []
    for x in range(im.width):
        for y in range(im.height):
            cs.append(im.getpixel((x,y)))
    return list(set(cs))

print("Original: %s" % colors(im))
print("Palette: %s" % RGBBW)
print("Convert: %s" % colors(cim))

Input image: -> <- (3x3 pixel image, all pixels unique colours)

(Larger version, for visualisation only: )

Output:

Original: [(85, 85, 85, 255), (0, 0, 255, 255), (0, 0, 0, 255), (255, 0, 0, 255), (0, 255, 255, 255), (255, 255, 255, 255), (255, 255, 0, 255), (255, 0, 255, 255), (0, 255, 0, 255)]
Palette: [(255, 0, 0), (0, 255, 0), (0, 0, 255), (0, 0, 0), (255, 255, 255)]
Convert: [(252, 252, 252), (0, 0, 255), (255, 0, 0), (0, 0, 0), (170, 170, 170), (0, 255, 0), (84, 84, 84)]

(Note that the hack to prevent dither is a workaround, pending a fix I've contributed to master (yet to be cut into a new release))

The values [(170, 170, 170), (84, 84, 84), (252, 252, 252)] appear in the converted image, but were not specified in the original palette. They all happen to be greyscale.

I think there's something in src/libImaging/Palette.c that's effecting this, but I'm not sure if this is a bug of the code, or a 'feature' of libjpeg

Does this help? https://stackoverflow.com/a/53482138/2836621 — Mark Setchell, Apr 01 '19 at 08:40
@MarkSetchell Input image added (with scaled up version that is actually visible.). The other question you link to was something I've already seen, it doesn't solve my issue, but it did help with fixing the dither bug [upstream](https://github.com/python-pillow/Pillow/pull/3699) — glasnt, Apr 01 '19 at 22:16
I think it's line 47 of `Palette.c` (that you link to) which is doing you in. It fills the unused entries in the palette with greyscale values and takes the liberty of using them! — Mark Setchell, Apr 02 '19 at 06:28
@MarkSetchell that's the bit! Not sure how you'd like credit for the solution, feel free to answer the question and I'll mark it answered. — glasnt, Apr 16 '19 at 16:19
I'm not too worried about credit - the main thing is to solve the problem. Feel free to write it up and accept your own answer. Post a link here if you get the PIL/Pillow team to make any changes or updates as a result. Good luck with your project! — Mark Setchell, Apr 16 '19 at 16:57

score 2 · Accepted Answer · answered Apr 19 '19 at 02:17

Turns out this issue is both user error and an unexpected initialisation issue.

The initialisation issue: As pointed out in the comments, the palette for a new image is specifically initialised as greyscale.

If we replace the entire palette with our own, then we're fine. Except, I wasn't.

data = sum([list(x) for x in RGBBW], [])[:256]

This line is logically incorrect.

The palette expects a flattened list of up to 256 triples of RGB, that is, an array of max len 768. If the array is anything less than this, then the rest of the greyscale will still be in play.

The better way to re-initialise the palette is to ensure we repeat a value as to override the greyscale.

In this case:

data = (sum([list(x) for x in RGBBW], []) + (RGBBW[-1] * (256 - len(RGBBW))))[:256*3]

That is:

data = (
    sum([list(x) for x in RGBBW], []) # flatten the nested array
    + (RGBBW[-1] * (256 - len(RGBBW))) # extend with the last value, to our required length, if needed
    )[:256*3] # and trim back, if needed.

This will result in the palette always being 768 length.

Using the last value from our provided array is an arbitrary choice, as is only used as a valid padding value.

Why does Pillow convert return colours outside the specified palette?

1 Answers1