Why image from a byte stream isn't being rendered?

Question

I'm working with the base64 module for image manipulation.

I've got this code:

import flask, base64, webbrowser, PIL.Image
...
...

image = PIL.Image.frombytes(mode='RGBA', size=(cam_width, cam_height), data=file_to_upload)
im_base64 = base64.b64encode(image.tobytes())
    html = '<html><head><meta http-equiv="refresh" content="0.5"><title>Displaying Uploaded Image</title></head><body><h1>Displaying Uploaded Image</h1><img src="data:;base64,{}" alt="" /></body></html>'.format(im_base64.decode('utf8'))
html_url = '/home/mark/Desktop/FlaskUpload/test.html'
with open(html_url, 'w') as f:
    f.write(html)
webbrowser.open(html_url)

I've also tried:

html = '<html><head><meta http-equiv="refresh" content="0.5"><title>Displaying Uploaded Image</title></head><body><h1>Displaying Uploaded Image</h1><img src="data:;base64,"'+im_base64.decode('utf8')+'" alt="" /></body></html>'

The heading is being rendered just fine but not the image. Have I missed anything ?

Update:

cam_width is 720

cam_height is 1280

file_to_upload is 3686400

first 10 bytes of the file_to_upload:

b'YPO\xffYPO\xffVQ'

I can't seem to get first 10 bytes of im_base64 with print(image.tobytes()[:10]) as it throws an error.

I got little bit closer to determining what's wrong. Once I fixed the quotes I got error:

Traceback (most recent call last):
  File "/home/mark/venv/lib/python3.7/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/mark/venv/lib/python3.7/site-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/home/mark/venv/lib/python3.7/site-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/mark/venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/mark/venv/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/mark/venv/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/mark/venv/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/mark/venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/mark/venv/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/mark/venv/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/mark/venv/server.py", line 28, in upload_file
    image = PIL.Image.frombytes(mode='RGBA', size=(cam_width, cam_height), data=file_to_upload)
  File "/home/mark/venv/lib/python3.7/site-packages/PIL/Image.py", line 2650, in frombytes
    im.frombytes(data, decoder_name, args)
  File "/home/mark/venv/lib/python3.7/site-packages/PIL/Image.py", line 797, in frombytes
    d.setimage(self.im)
ValueError: tile cannot extend outside image

I'm working with image manipulation for the very first time so I don't know what I'm doing. What ValueError: tile cannot extend outside image means ?

You have omitted some details that will help to debug your code. What is the value of `cam_width` and `cam_height` and what is the length in bytes of `file_to_upload`? What are the first 10 bytes of `file_to_upload`? What are the first 10 bytes of `im_base64`? — Mark Setchell, Sep 22 '20 at 16:25
Could be related: https://stackoverflow.com/questions/25140826/generate-image-embed-in-flask-with-a-data-uri — รยקคгรђשค, Sep 22 '20 at 17:05

Mark Setchell · Accepted Answer · 2020-09-24T09:25:45.297

To see where you are going wrong, you need to differentiate between:

RGB "pixel data", and
JPEG/PNG encoded images.

"Pixel data" is a bunch of RGB/RGBA bytes and that is all. There is no height or width information to know how to interpret or lay out the pixels on a screen. The data is just 4 RGBA bytes for each pixel. If you know your image is 720x1280 RGBA pixels, you will have 720x1280x4, or 3686400 bytes. Notice there's no room in there for height and width or the fact it's RGBA. That's what you have in your variable file_to_upload. Note that you had to additionally tell PIL Image the height and width and the fact it is RGBA for PIL to understand the pixel data.

A JPEG/PNG encoded image is very different. Firstly, it starts with a magic number, which is ff d8 for JPEG, and the 3 letters PNG and some other bits and pieces for PNG. Then it has the height and width, the bytes/pixel and colourspace and possibly the date and GPS location you took the photo, your copyright, the camera manufacturer and lens and a bunch of other stuff. Then it has the compressed pixel data. In general, it will be smaller than the corresponding pixel data. A JPEG/PNG is self-contained - no additional data is needed.

Ok, you need to send a base64-encoded JPEG or a PNG to the browser. Why? Because the browser needs an image with dimensions in it, else it can't tell if it is 720 px wide and 1280 px tall, or a single straight line of 921,600 RGBA pixels, or a single straight line of 1,228,800 RGB pixels. Your image is RGBA, so you probably better send a PNG because JPEGs cannot contain transparency.

So, where did you go wrong? You started with "pixel data", added in your knowledge of height and width and made a PIL Image. So far so good. But then you went wrong because you called tobytes() and made it back into exactly what you started with - "pixel data" with the same length and content as you had, and no width or height info. You should have instead, created an in-memory PNG-encoded image with the height and width embedded in it so that the browser knows its shape. Then base64-encode and send that. So you needed something like:

image = PIL.Image.frombytes(mode='RGBA', size=(cam_width, cam_height), data=file_to_upload)
buffer = io.BytesIO()
image.save(buffer, format="PNG")
PNG = buffer.getvalue()

Also, have a read here about checking the first few bytes of your data so you can readily check if you are sending the right thing.

So, here's the complete code:

#!/usr/bin/env python3

import base64
import numpy as np
from PIL import Image
from io import BytesIO

cam_width, cam_height = 640, 480

# Simulate some semi-transparent red pixel data
PixelData = np.full((cam_height,cam_width,4), [255,0,0,128], np.uint8)

# Convert to PIL Image
im = Image.frombytes(mode='RGBA', size=(cam_width, cam_height), data=PixelData)

# Create in-memory PNG
buffer = BytesIO()
im.save(buffer, format="PNG")
PNG = buffer.getvalue()

# Base64 encode
b64PNG = base64.b64encode(PNG).decode("utf-8") 

# Create HTML
html = f'<html><head><meta http-equiv="refresh" content="0.5"><title>Displaying Uploaded Image</title></head><body><h1>Displaying Uploaded Image</h1><img src="data:;base64,{b64PNG}" alt="" /></body></html>'

# Write HTML
with open('test.html', 'w') as f:
    f.write(html)

And the resulting, semi-transparent red image:

The reason why I'm converting image from bytes on one line and then converting it back to bytes on the next line is that I'm rotating the image by 90 degrees between the two lines. I just didn't include it in the post as I didn't find it relevant. — Mark, Sep 24 '20 at 09:52
Very well explained. I took notes as well. Since I'm developing an app that deals with image manipulation quite a bit could you recommend me resources that will help me gain good understanding about how Python handles images and different types of encodings ? In other words where did you gain a good understanding of this topic ? — Mark, Sep 24 '20 at 10:06
Did you simply play around with Python modules such as `io`, `base64`, `PIL` long enough to get hang of it ? I'm all ears here. — Mark, Sep 24 '20 at 10:16
I did a Masters in Satellite Imaging. For me, the easiest way to learn was with **ImageMagick** because you can do almost anything you want with images just using the command line in Terminal without writing any code. Anthony Thyssen's pages are **extremely useful** at explaining how to do everything from cropping through filtering and morphology https://www.imagemagick.org/Usage/ and the detailed online help on every single command is here https://imagemagick.org/script/command-line-options.php — Mark Setchell, Sep 24 '20 at 10:29

Why image from a byte stream isn't being rendered?

1 Answers1