6

I have a set of 24-bit png files and I want to transform them into 8-bit png files. I used PIL's Image.convert() method for solving this problem. However, after using mode 'P' as the argument, I found out that pixels with same RGB values can be converted differently.

I transferred an example image into a numpy array and the original 24-bit png file has values like this:

RGB array

   ..., 
   [204, 102, 119],
   [204, 102, 119],
   [204, 102, 119],
   [204, 102, 119],
   [204, 102, 119],
   [204, 102, 119],
   [204, 102, 119],
   [204, 102, 119],
   [204, 102, 119],
   [204, 102, 119],
   [204, 102, 119], 
   ...

After using the convert function with mode 'P', the images value became like this:

8-bit array

   ..., 98, 98, 134, 98, 98, 98, 134, 98, 98, 98, 134, ...

Code Example:

from PIL import Image
import numpy as np
img = Image.open("path-to-file.png")
p_img = img.convert("P")

I expect that pixels with same RGB values are converted in the same way. I know pixels are converted into palette index, but this still doesn't make sense to me. I'm not familiar with the PIL library. Can someone please explain why this happens? Thanks in advance.

Implemented a little something following Mark's examples

import numpy as np
from PIL import Image
#from improc import GenerateNColourImage

# Set image height and width
N    = 6
h, w = 100, 100

# Generate repeatable random Numpy image with N^3 unique colours at most
n = np.random.randint(N, size=(h, w, 3), dtype=np.uint8)
# Work out indices of diagonal elements
diags = np.diag_indices(h)

# Intentionally set all diagonal elements same shade of blue
n[diags] = [10,20,200]
# Make Numpy image into PIL Image, palettise, convert back to Numpy array and check diagonals
a0 = Image.fromarray(n)

unique_colors = np.unique(n.reshape(-1, n.shape[2]), axis=0).shape
print(unique_colors)   #e.g. print (217, 3)
a1 = a0.convert('P')
a2 = np.array(a1)

# Look at diagonals - should all be the same
print(a2[diags])
print(' %s %d' % ("Number of unique colors:  ", np.unique(a2).shape[0]))

Diagonal pixels' values printed

... 154 154 154 154 154 154 124 154 160 160 160 154 160 ...

The 8-bit image in mode 'P' contains 125 unique palette indexes. It seems PIL will perform dithering no matter what.

VicXue
  • 189
  • 1
  • 2
  • 8

2 Answers2

2

This is a normal behavior shown, when we convert a Image into P color mode. The way Palette mode works is it creates a mapping table, which corresponds a index (in range 0 - 255) to a discrete color in larger color space (like RGB). For example, RGB color value (0, 0, 255) (Pure Blue) in an image gets an index 1 (just an hypothetical example). This same process goes through each unique pixel value in the original image (but the table size should not exceed 256, in the process of mapping). So, the numpy array (or a regular list) having values like this:-

   ..., 98, 98, 134, 98, 98, 98, 134, 98, 98, 98, 134, ...

corresponds to the index, in the mapping table, rather then the actual color value itself. So, you may interpret them as a index, which upon reading an image gets converted to the actual color value stored in that index.

But these pixel values, need not always mean the image is of color mode P. For example, if you view the pixel data of a Greyscale image (L), the values would look the same like in the case of paletted mode, but would actually correspond to true color values (or shades of grey), rather then a index.

Vasu Deo.S
  • 1,820
  • 1
  • 11
  • 23
  • Hi Vasu, thank you for this reply. I understand that RGB values get mapped into a palette index, but I was not sure why same RGB values can have different indexes. It might be dithering, as Mark said in his answer, for a better visualization performance. Additionally, out of this topic, do you know how a picture tells whether it is a grayscale image or a palette image? – VicXue Jul 10 '19 at 23:47
  • @VicXue Well, if you are asking how to recognize image color mode in *PIL*, then you have to use `img.mode` (where `img` is the name of the image object). This will give you the color mode of the opened image. (where color mode = {RGB, P, 1, L, I etc}) – Vasu Deo.S Jul 11 '19 at 02:19
  • No, I mean if both _L_ and _P_ mode png files are in 8-bit format, how do image viewers tell the difference between them and visualize them differently? – VicXue Jul 11 '19 at 02:58
  • @VicXu You never know. A paletted image, could be same as a greyscale one. But the reverse is not true. By that i mean, if seen at a visual level (not going through image properties/metadata) a `P` image, having shades of grey as its mapping table, will look similar to a greyscale version of that image. The reason is because greyscale sample space belongs to the range 0 - 255 (or 256 unique values), and `P` image mode is also capable of storing that much unique colors. So, a greyscale image could very well be converted to a `P`, and the difference is almost negligable. – Vasu Deo.S Jul 11 '19 at 03:16
  • @VicXue The reverse is not true, as greyscale image could not hold color values (unlike a paletted image). So that could be used as a distinction atleast at a visual level, that if an image contains any form of RGB color (or color outside Grey range), then it is definitely a Paletted image (if only two image exists one being paletted and other being greyscale). But, if a greyscale image, is used, the distinguishing process is not that easy. (Atleast at a visual level) as the difference between the two images is imperceptible. – Vasu Deo.S Jul 11 '19 at 03:19
  • 1
    @Mark has given a really good [answer](https://stackoverflow.com/questions/52307290/what-is-the-difference-between-images-in-p-and-l-mode-in-pil/52307690#52307690), on this topic. It may not be the same as what you asked, but still contains a lot of information regarding this. – Vasu Deo.S Jul 11 '19 at 03:22
  • Thank you for your detailed explanation. I ought to choose my words more carefully so that you can understand me better. I know human eyes are able to tell the difference between two images in different modes only if one of the images (in mode 'P') has colors other than just grey. However, I'm wondering how computers tell the difference between two kinds of images. If these two types of images are consisted by a series of 8-bit values only, how do computers know they should be visualized differently? This might be a naive question to you, but I'm not familiar with computer vision. – VicXue Jul 11 '19 at 05:55
  • @VicXue Well, computers have access to what's called "*Metadata*", meaning data related to data. A image color mode/space isn't recognized by it's pixel values, rather it is determined by what its metadata states. I am myself unsure of the exact details of the metadata, i have asked a [question](https://stackoverflow.com/questions/56965791/how-to-recognize-an-image-file-format-using-its-contents) regarding the metadata of image files (it is currently unanswered), which closely relates to your query. Once that question has been answered, the answer will help you a lot too. ^_^ – Vasu Deo.S Jul 11 '19 at 06:36
2

The issue is that PIL/Pillow is "dithering". Basically, if you have more than the 256 colours (maximum a palette can hold) in your image, there are necessarily colours in the image that do not occur in the palette. So, PIL accumulates the errors (difference between original colour and palettised colour) and every now and then inserts a pixel of a slightly different colour so that the image looks more or less correct from a distance. It's basically "error diffusion". So, if your colour gets caught up in that, it will sometimes come out differently.

One way you can avoid that, is to quantise the image down to less than 256 colours, then there will be no errors to diffuse.

# Quantise to 256 colours
im256c = = image.quantize(colors=256, method=2)

Note that this does not mean your shade of blue will always map to the same palette index in every image, it just means all pixels with your shade of blue in any one given image will all have the same palette index.

Here is an example:

#!/usr/bin/env python3

import numpy as np
from PIL import Image
from improc import GenerateNColourImage

# Set image height and width
h, w = 200, 200
N    = 1000

# Generate repeatable random Numpy image with N unique colours
np.random.seed(42)
n = GenerateNColourImage(h,w,N) 

# Work out indices of diagonal elements
diags = np.diag_indices(h)

# Intentionally set all diagonal elements same shade of blue
n[diags] = [10,20,200]

# Make Numpy image into PIL Image, palettise, convert back to Numpy array and check diagonals
a0 = Image.fromarray(n)
a1 = a0.convert('P')
a2 = np.array(a1)

# Look at diagonals - should all be the same
print(a2[diags])

Output 

[154 154 154 154 154 154 154 154 160 154 160 154 154 154 154 154 160 154
 154 154 160 154 154 154 160 160 154 154 154 160 154 154 154 154 154 154
 154 154 160 154 154 154 154 154 154 154 154 160 160 154 154 154 154 154
 154 154 154 154 154 154 154 154 154 160 154 154 154 154 154 154 154 160
 154 160 154 154 154 154 154 154 154 154 154 154 154 160 154 160 160 154
 154 160 154 154 154 160 154 154 154 154 154 160 154 154 154 154 155 154
 154 160 154 154 154 154 154 154 154 154 154 160 154 154 154 160 154 154
 154 154 160 154 154 154 154 154 154 154 154 154 154 154 154 154 154 160
 154 160 154 160 154 160 154 160 160 154 154 154 154 154 154 154 154 154
 154 154 154 154 161 154 154 154 154 154 154 154 154 154 154 160 154 160
 118 154 160 154 154 154 154 154 154 154 154 154 160 154 154 160 154 154
 154 154]

Ooops, there are values of 154, 118 and 160 in there...

Now do those last 4 lines again, with the same Numpy Array, but using quantise():

# Make Numpy image into PIL Image, quantise, convert back to Numpy array and check diagonals
b0 = Image.fromarray(n)
b1 = b0.quantize(colors=256,method=2)
b2 = np.array(b1)

# Look at diagonals - should all be the same
print(b2[diags])

Output

[64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64
 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64
 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64
 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64
 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64
 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64
 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64
 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64
 64 64 64 64 64 64 64 64]

That's better - all the same!

I should add, that if you save an image as PNG with 256 colours or less, PIL/Pillow will automatically save a palette image.

Mark Setchell
  • 191,897
  • 31
  • 273
  • 432
  • I am aware of Floyd–Steinberg dithering (the one which is used if we convert an image into `1`). It is used to give a perception (or feel) that an image contains more colors, then what it actually contains (used in *B&W* image, to make it look greyscale if viewed from a distance). Is error diffusion dithering the same? – Vasu Deo.S Jul 10 '19 at 08:38
  • Secondly, can you please elaborate on the point *PIL accumulates the errors (difference between original colour and palettised colour) and every now and then inserts a pixel of a slightly different colour*. I am unable to understand how are colors out of the 256 palette range stored/inserted every now and then? – Vasu Deo.S Jul 10 '19 at 08:39
  • Thirdly, isn't this problem same for every format that uses **color indexing**? Then why *The issue is that PIL/Pillow is "dithering"*, isn't this something that almost all color indexing format would use in one way or other, in order to compensate for color outside the palette range? – Vasu Deo.S Jul 10 '19 at 08:42
  • I can understand why PIL performs dithering when there are more unique colors in an image than the palette map can hold. However, in my case, I only have two unique RGB values in my images, as they are just binary masks that differentiate between background and an interested object. I also implemented your code with 217 unique RGB values. After using the convert function with mode 'P', the diagonal pixels still vary in values. Does this mean convert function with mode 'P' perform dithering no matter what? – VicXue Jul 10 '19 at 23:25
  • By the way, the quantise() method does work very well in my case. Thank you so much for that. – VicXue Jul 10 '19 at 23:49
  • @VicXue Would you be able to share your image please, so I can get to the bottom of this? – Mark Setchell Jul 11 '19 at 05:58
  • @Mark Here is a link to one of the example images [link](https://ibb.co/rt71PZh) Maybe you can also take a look at my example codes above? Thank you. – VicXue Jul 11 '19 at 23:35