Trouble creating PIL Image object from canvas tag with Selenium

Question

I am trying to create a PIL Image object based on a canvas tag extracted with Selenium from this website. The goal is to use pytesseract and get the captcha content. My code doesn't raise any errors, but the image created is all black.

My code so far:

# Run JS code to get data URI
png_url = driver.execute_script(
        'return document.getElementsByTagName("canvas")[0].toDataURL("image/png");')
# Parse the URI to get only the base64 part
str_base64 = re.search(r'base64,(.*)', png_url).group(1)
# Convert it to binary
str_decoded = str_base64.decode('base64')
# Create and show Image object
image = Image.open(StringIO(str_decoded))
image.show()    
# Read image with pytesseract
recaptcha = pytesseract.image_to_string(image)

I don't know why the image is all black. My code is based on this tutorial, which saves the image. I dont want to save the image, I want it to be in memory only.

Edit:

I have saved the image in filesystem and the image is saving ok, but with transparent background, thus appearing black when showed. How can I make the background white?

score 1 · Answer 1 · edited May 23 '17 at 11:55

All I needed to do is extract the background following this answer:

def remove_transparency(im, bg_colour=(255, 255, 255)):

    # Only process if image has transparency (https://stackoverflow.com/a/1963146)
    if im.mode in ('RGBA', 'LA') or (im.mode == 'P' and 'transparency' in im.info):

        # Need to convert to RGBA if LA format due to a bug in PIL (https://stackoverflow.com/a/1963146)
        alpha = im.convert('RGBA').split()[-1]

        # Create a new background image of our matt color.
        # Must be RGBA because paste requires both images have the same format
        # (https://stackoverflow.com/a/8720632  and  https://stackoverflow.com/a/9459208)
        bg = Image.new("RGBA", im.size, bg_colour + (255,))
        bg.paste(im, mask=alpha)
        return bg

    else:
        return im

The complete code then:

png_url = driver.execute_script(
            'return document.getElementsByTagName("canvas")[0].toDataURL("image/png");')
str_base64 = re.search(r'base64,(.*)', png_url).group(1)
# Convert it to binary
str_decoded = str_base64.decode('base64')
image = Image.open(StringIO(str_decoded))
image = remove_transparency(image)
recaptcha = pytesseract.image_to_string(image).replace(" ", "")

score 0 · Answer 2 · edited May 23 '17 at 10:29

0

You should create a RGB white image and paste your RGBA image into it. Solution may be this, but there are other ways too. I suggest numpy and opencv.

edited May 23 '17 at 10:29

Community

1
1

answered Jun 21 '16 at 16:45

danielarend

1,379
13
26

Trouble creating PIL Image object from canvas tag with Selenium

2 Answers2