Using numpy and pil to convert 565(16bit-color) to 888(24bit-color)

Question

I must preface this, with the fact that I have a working method using bitshift and putpixel, but it is incredibly slow, and I am looking to leverage numpy to speed up the process. I believe I am close, but not quite there. Having timed what I think should work, I'm seeing a 0.3 second improvement in time, hence my motivation.

The current working code:

buff # a binary set of data
im = Image.new("RGBA",(xdim,ydim))
for y in xrange(ydim):
    for x in xrange(xdim):
        px = buff[x*y]
        # the 255 is for the alpha channel which I plan to use later
        im.putpixel((x,y),(px&0xF800) >> 8, (px&0x07E0) >> 3, (px&0x001F) <<3, 255))
return im

The code I'm trying to get work looks like this:

im16 = numpy.fromstring(buff,dtype=numpy.uint16) #read data as shorts
im16 = numpy.array(im16,dtype=numpy.uint32) #now that it's in the correct order, convert to 32 bit so there is room to do shifting
r    = numpy.right_shift(8, im16.copy() & 0xF800)
g    = numpy.right_shift(3, im16.copy() & 0x07E0)
b    = numpy.left_shift( 3, im16 & 0x001F)
pA   = numpy.append(r,g)
pB   = numpy.append(b,numpy.ones((xdim,ydim),dtype=numpy.uint32) * 0xFF) #this is a black alpha channel
img  = numpy.left_shift(img,8) #gives me green channel
im24 = Image.fromstring("RGBA",(xdim,ydim),img)
return im24

so the final problem, is that the channels are not combining and I don't believe I should have to do that final bit shift (note that I get the red channel if I don't bit-shift by 8). Assistance on how to combine everything correctly would be much appreciated.

SOLUTION

import numpy as np
arr = np.fromstring(buff,dtype=np.uint16).astype(np.uint32)
arr = 0xFF000000 + ((arr & 0xF800) >> 8) + ((arr & 0x07E0) << 5) + ((arr & 0x001F) << 19)
return Image.frombuffer('RGBA', (xdim,ydim), arr, 'raw', 'RGBA', 0, 1)

the difference is that you need to pack it as MSB(ALPHA,B,G,R)LSB counter intuitive from putpixel, but it works, and works well

Justin Peel · Accepted Answer · 2011-03-24T15:18:32.297

3

Warning: the following code hasn't been checked, but I think that this will do what you want (if I'm understanding everything correctly):

import numpy as np
arr = np.fromstring(buff,dtype=np.uint16).astype(np.uint32)
arr = ((arr & 0xF800) << 16) + ((arr & 0x07E0) << 13) + ((arr & 0x001F) << 11) + 0xFF
return Image.frombuffer('RGBA', (xdim,ydim), arr, 'raw', 'RGBA', 0, 1)

I'm combining all of the channels together into 32-bits on the line that does all of the bit shifting. The leftmost 8-bits are the red, the next 8 are the green, the next 8 blue, and the last 8 alpha. The shifting numbers may seem a little strange because I incorporated the shifts from the 16-bit format. Also, I'm using frombuffer because then we want to take advantage of the buffer being used by Numpy rather than converting to a string first.

It might help to look at this page. It's not super great in my opinion, but that's how things go with PIL in my experience. The documentation is really not very user-friendly, in fact I often find it confusing, but I'm not about to volunteer to rewrite it because I don't use PIL much.

edited Mar 24 '11 at 15:18

answered Mar 24 '11 at 06:02

Justin Peel

46,722
6
58
80

Thanks for the quick response. That looks correct, but when I tried it, it did not work correctly. I put in some debug statements ... beginning of buffer looks like ['\x10\x00\x0b\xe5'] ... now I did a dump of a 'short' buffer to 'char' buffer so endian-ness got reversed. However, the conversion seems to have worked in my favor since looking at arr[0] => \x10L and arr[1] => \xe50b (this would be after the second line in your snippet. Whats really puzzling is that after the 3rd line, I again looked at arr[0] and was surprised to see 0x0L ... at the very least, I would have expected 0xFF. – pyInTheSky Mar 24 '11 at 06:48
Interesting, it seems that the problem was in wrapping the <<'s in parens as well. The '+' operator seems to take precedence! – pyInTheSky Mar 24 '11 at 06:57
Everything looks good except that for some reason, the colors are off. I looked on PIL documentation, but didn't see a list of what you can replace 'raw' or 0,1 with .. i'm guessing that the 0,1 has to do w/ position, but is there anything you can think of that would cause funky colors? .. Thanks : ) (I've experimented w/ all the bit shift combos I can think of) – pyInTheSky Mar 24 '11 at 07:45
That's weird about the <<'s needing to be in parentheses as well, but I put them in. Actually, looking at http://docs.python.org/reference/expressions.html#summary I guess it makes sense. I didn't realize that Python had different operator precedence than C with the shifting operators, but then again I don't use them very often. – Justin Peel Mar 24 '11 at 15:21
I'm not surprised that the colors are weird. You're taking numbers on one scale and putting those values only in the upper parts of another scale. In other words, you need to scale the values. You'll probably need floating point to get it very accurate. You might want to have PIL do the conversion for you as in this question: http://stackoverflow.com/questions/4199497/image-frombuffer-with-16-bit-image-data, but if you keep with what we've been doing, you will have more control. – Justin Peel Mar 24 '11 at 15:33
so close, finally got the solution (posted in original post) thanks so much for the help – pyInTheSky Mar 25 '11 at 02:33
glad I could help. That is strange, but at least you've solved it. – Justin Peel Mar 25 '11 at 03:45

score 1 · Answer 2 · answered Jun 22 '11 at 03:11

If you want to do the scaling appropriately, here is a more PIL-ish to solve your problem.

FROM_5 = ((np.arange(32, dtype=numpy.uint16) * 255 + 15) // 31).astype(numpy.ubyte)
FROM_6 = ((np.arange(64, dtype=numpy.uint16) * 255 + 31) // 63).astype(numpy.ubyte)

data = numpy.fromstring(buff, dtype=numpy.uint16)
r = Image.frombuffer('L', shape, FROM_5[data >> 11], 'raw', 'L', 0, 1)
g = Image.frombuffer('L', shape, FROM_6[(data >> 5) & 0x3F], 'raw', 'L', 0, 1)
b = Image.frombuffer('L', shape, FROM_5[data & 0x1F], 'raw', 'L', 0, 1)
return Image.merge('RGB', (r, g, b))

I don't believe the posted 'solution' has a scaling problem. Could you point out where the flaw exists? Thanks : ) — pyInTheSky, Jun 27 '11 at 20:27

Using numpy and pil to convert 565(16bit-color) to 888(24bit-color)

SOLUTION

2 Answers2

Linked