Converting an autopy bitmap to a Pillow image

Question

I'm developing a screen scraper in Python, using Autopy and Pillow.

Is it possible to convert a bitmap object to a Pillow image object?

My current solution is to save the bitmap object as an image file, and then use the path to create a Pillow image object. This approach is really slow because of harddrive I/O.

My current (very slow) solution:

from PIL import Image
import autopy

bitmap_object = autopy.bitmap.capture_screen()
bitmap_object.save('some/path.png') # VERY SLOW!
img = Image.open('some/path.png')

Question: Can the above functionality be achieved without saving the bitmap object to the harddrive?

Try using `ImageGrab` from `PIL` which does the same thing as `autopy.bitmap.capture_screen()`. Link [here](http://pillow.readthedocs.io/en/3.1.x/reference/ImageGrab.html). — Vasilis G., Dec 13 '17 at 16:23
That wont work, because Im using the autopy bitmap for many things, including searching for pixel patterns. I only need the PIL image for OCR. — Vingtoft, Dec 13 '17 at 16:25
Can you try running it using PyPy instead of python and see if it helps? Not sure if PyPy support PIL and autopy but you can give it a shot — Tarun Lalwani, Dec 13 '17 at 16:46
Have you tried using `img = PIL.Image.fromArray(bitmap, mode=xx)` This works for me with numpy arrays. I see Autopy.bitmap has a `to_string()` method. Not sure if that'll get you the raw data or not. — bivouac0, Feb 02 '18 at 22:10
@bivouac0 the `to_string` method has no documentation on the format it produces. I'd love to help but I can't seem to install `autopy`, pip keeps returning errors. — Mark Ransom, Feb 02 '18 at 22:33

score 2 · Answer 1 · answered Feb 02 '18 at 22:41

After looking through the source code it looks like there is not a way to directly access the raw bitmap. However, you can get an encoded copy.

First, get its encoded representation.

bitmap_encoded = bitmap_object.to_string()

This is encoded as "b", followed by the width, comma, height, comma, and a base64 encoding of zlib compressed raw bytes. Parse the encoded data:

import base64
import zlib

# b3840,1080,eNrsf...H1ooKAs=
#      ^    ^
first_comma = bitmap_encoded.find(',')
second_comma = bitmap_encoded.find(',', first_comma + 1)

# b3840,1080,eNrsf...H1ooKAs=
#  ^  ^
width = int(bitmap_encoded[1:first_comma])

# b3840,1080,eNrsf...H1ooKAs=
#       ^  ^
height = int(bitmap_encoded[first_comma+1:second_comma])

# b3840,1080,eNrsf...H1ooKAs=
#            ^
bitmap_bytes = zlib.decompress(base64.b64decode(bitmap_encoded[second_comma+1:]))

When I tested this on my machine, the red and blue channels were backwards so I'm assuming the bitmap from autopy is RGB encoded instead of the typical BGR encoding that BMP files use which is what PIL expects. Finally, load the image with PIL:

img = PIL.Image.frombytes('RGB', (width, height), bitmap_bytes, 'raw', 'BGR', 0, 1)

To load the image normally without swapping the red and blue channels, do:

img = PIL.Image.frombytes('RGB', (width, height), bitmap_bytes)

Did you happen to try iterating on the object directly? (ie... `for x in bitmap_object`) I see there's a `Bitmap_iternext` that I thought might be usable to get a raw array, although I don't have this library installed here to mess with it. — bivouac0, Feb 02 '18 at 22:50
@bivouac0 I did but it's not really useful. It appears to just yield the points. I.e., `iter(bitmap_object).next() == (1, 0)` and `list(bitmap_object) == [(1, 0), (2, 0), ..., (3838, 1079), (3839, 1079)]`. — Uyghur Lives Matter, Feb 02 '18 at 22:55
Awesome solution, works perfectly, just what I was hoping for. Thanks! — Vingtoft, Feb 06 '18 at 08:20

score 0 · Answer 2 · answered Oct 11 '20 at 22:24

Looks like now this has a solution from autopy:

import autopy
import PIL.Image

bmp = autopy.bitmap.capture_screen()
width, height = int(round(bmp.width * bmp.scale)), int(round(bmp.height * bmp.scale))
img = PIL.Image.frombytes('RGB', (width, height), bytes(bmp))

Converting an autopy bitmap to a Pillow image

2 Answers2

Linked