Converting an encoded JPEG image to Uint8Array in JavaScript

Question

I have a python server that sends a frame of RGB data to my JS script via a websocket. The frame is encoded using simplejpeg python package. The code looks like this:

jpeg_frame = simplejpeg.encode_jpeg(image=color_frame, quality=85,colorspace='RGB', colorsubsampling='444', fastdct=True)

jpeg_frame is passed to the websocket and sent to the JS script.

On the JS side however, I would like to decompress the image and have it in the form of a Uint8Array so that I can work with the data. The image does not have to be viewed.

The data recieved is in the form of ArrayBuffer.

This is what I have so far.

socketio.on('colorFrame',(data)=>{
    var mime = 'image/jpeg';
    var a = new Uint8Array(data);
    var nb = a.length;
    if (nb < 4)
        return null;
    var binary = "";
    for (var i = 0; i < nb; i++)
        binary += String.fromCharCode(a[i]);
    var base64 = window.btoa(binary);
    var image = new Image();
    image.src = 'data:' + mime + ';base64,' + base64;
    
    var canvas = document.createElement('canvas');
    var ctx = canvas.getContext('2d');

    ctx.drawImage(image,0,0)

    image.onload = function(){
        var image_data = ctx.getImageData(0,0, 960, 540);
        console.log(image_data);
    };
});

So far I could not yet figure out how I can decompress the image. I dont mind the inaccuracy of the lossy compression, I just want the image back to its original resolution and be able to convert it to a Uint8Array.

What is the simplest way to get JPEG decoding working in a JS scrip?

You might want to use https://www.npmjs.com/package/base64-arraybuffer instead of rolling your own decoding... — AKX, Jun 14 '21 at 07:54
But yep, you're on the right track. Once you have a loaded `Image`, you can use a canvas 2D context to paint it, then `getImageData` to work with the pixels. — AKX, Jun 14 '21 at 07:55
You will also want to attach to the `image.onload` event, as even with data uri the image is not ready until onload is fired. — Keith, Jun 14 '21 at 08:07
Thanks for the advice. I have updated the answer to include the new lines of code. When I console log the image data, all the pixels are 0. — Paul Brink, Jun 14 '21 at 09:23

score 1 · Answer 1 · answered Nov 27 '21 at 16:52

There is an excellent article "Decoding a PNG Image in JavaScript" that answers the literal question. It's a non-trivial answer. (Note: I am not the author of the article)

However

Based on your code, it looks like you might not care about actually decoding the Image yourself. If you just want to display it, take a look at this StackOverflow question

Converting an encoded JPEG image to Uint8Array in JavaScript

1 Answers1

However