Decode image bytes data stream to JPEG

Question

I am struggling to successfully decode a JPEG image from bytes, back to JPEG again.

I started from encoded frame from a MJPG bytes stream, which I want to decode in order to manipulate with OpenCV. I am a bit of a newbie at Python, numpy, opencv etc!

I now have the frame JPG data in a text file as: b'\xf\xd8\xff\xdb\x00....etc etc for purposes of testing:

code seems to fail when I try to Resize the numpy array to the original video stream resolution (640, 480) on line 14 (npFlat.reshape((640,480))

**ValueError: cannot reshape array of size 228140 into shape (640,480)*

import io
import cv2
import numpy as np

BytesFile = open('FrameBytes.txt')
MyBytes=BytesFile.read()
BytesFile.close()

dt=np.dtype(np.unit8)
dt=dt.newbtyeorder('>')

npFlat = np.fromfile('FrameBytes.txt'.dtype=dt)
npResized = npFlat.reshape(640,480,3) #CODE FAILING TO RESIZE AT THIS LINE...
cv.imshow('resized',npResized)

Could it be that even though my video frame was captured from a 640, 480 feed, for some reason during encoding the size has changed? This is all I can think of at the moment. Any/all help welcome.

I have reviewed a related post:Python - byte image to NumPy array using OpenCV but trying to avoid PIL, and the frombuffer method also seems to be failing for me.

Ok, so I made some progress and now have:

npFlat = np.frombuffer(MyBytes.encode('utf-8'),dtype=np.int8).

I can now also get the code to succeed when I 'reshape' npFlat to (374, 610). I.e. so that 374 x 610 = the flat numpy array, which is of length 228140...but this all seems odd. the buffer information represents a JPG which I am trying to reopen...and am not getting close yet.

MyBytes.txt" Data Input File is viewable here: https://drive.google.com/file/d/18pqILl9myeTRjdiqtExFyOe94Km_aNNM/view?usp=sharing]1

Answer 1

You have made quite a mess - you should avoid saving binary data as text files!

Copy the contents of your file into your clipboard - on a Mac the command is:

cat frame.txt | pbcopy

Start Python and make a variable called s and set it to the contents of the clipboard:

s = PASTE_YOUR_CLIPBOARD

Now do:

from PIL import Image
from io import BytesIO

# Load image from BytesIO
im = Image.open(BytesIO(s))

# Display image and save image
im.show()
im.save('result.png')

If you are on OpenCV, use:

import cv2

# Make s as above
s = PASTE_YOUR_CLIPBOARD

i = np.frombuffer(s,dtype=np.uint8)

im = cv2.imdecode(i,cv2.IMREAD_UNCHANGED)

cv2.imwrite('result.png',im)

Answer 2

Your FrameBytes.txt file, despite the extension is really a JPG file (unless you garbled it on saving anyway).

So, you have t read it with a proper image reader, not as bytes, if you want to manipulate it as an image. Otherwise, you have the JPEG encoded bytes - your 228140 bytes on disk represent an image with 921600 bytes of data when decompressed. (which is reasonable for a high-quality jpeg file).

Simply use PIL for that:

from PIL import Image

img = Image.open("FrameBytes.txt")
print(img.size) # this should print (640, 480).

# to convert it to a numpy array:
import numpy as np
data = np.array(list(img.tobytes()), dtype="uint8")
data.shape = img.size + (3,)

...

If you don't want to use PIL, of course,you can use other libraries - opencv itself have the cv2.imread method that should work out of the box.