5

I am using headless-gl to run webGL on Node.js, creating an image dynamically on server. Once created, the image is to be stored in database (MongoDB), before user accessing the image again via API.

Below is the part where the image is generated:

var pixels = new Uint8Array(width * height * 4)
gl.readPixels(0, 0, width, height, gl.RGBA, gl.UNSIGNED_BYTE, pixels)

The pixels are then converted to base64 (since this seems to be the recommended way to be loaded by Image in client HTML).

var base64Image = new Buffer(pixels, 'binary').toString('base64');

However, the string produced by this buffer can't be decoded to produce an image. May be pixels is not 'binary' type? Or should I just save the pixel string in the database and try to redraw the pixel in canvas pixel by pixel in client (I don't think this is the best way) ?

kevinadi
  • 13,365
  • 3
  • 33
  • 49
cristie09
  • 311
  • 1
  • 3
  • 11
  • What you mean by "read back to produce an image"? I mean the code. – Jake Nov 01 '17 at 14:25
  • I've edited 'read back' to decoded, hope it clarifies! – cristie09 Nov 01 '17 at 15:45
  • Below is the part where the image is generated: No this is not generated image it is image data ( color data per pixel ) . This data can be used with frameBuffer . It's not easy . Did your base image file comes from your server( domain ) if not you can't generate image for sure. Did you try to debug variable pixels and see what is it ? – Nikola Lukic Nov 02 '17 at 14:29
  • hi, the variable pixels contains the color data per pixel of the image I have generated previously (code not attached). I would like to make this variable pixels into a string so that the image can be saved in database, before it is downloaded by client's browser. Would like to know how to convert it and the right format to use -- seems like base64 is recommended after some initial search I did online – cristie09 Nov 02 '17 at 18:59

2 Answers2

2

What do you get from gl.readPixels is bitmap data. I use Jimp library to convert the data to different format and then get base64 image string from it.

NodeJS code

var Jimp = require("jimp")

//Create context
var width = 50
var height = 50

var gl = require('gl')(width, height, { preserveDrawingBuffer: true })

//Clear screen to red
gl.clearColor(1, 0, 0, 1)
gl.clear(gl.COLOR_BUFFER_BIT)

// get bitmap data 
var bitmapData = new Uint8Array(width * height * 4)
gl.readPixels(0, 0, width, height, gl.RGBA, gl.UNSIGNED_BYTE, bitmapData)

// use Jimp library to create an image with a specific mime type
var image = new Jimp(width, height, function (err, image) {

    // assign the bitmap as data for the image 
    image.bitmap.data = bitmapData
    // generate base64
    image.getBase64("image/png", function (error, str) {
        // result
        console.log(str)
    })
})

Test in the browser if it works for you (JS code):

var base64Img = "" // TODO paste the result from NodeJS

document.body.innerHTML = ""
document.write('<img src="' + base64Img + '"/>');
Entity Black
  • 3,401
  • 2
  • 23
  • 38
1

What you get using readPixels are raw bitmap data, if your goal is to have this displaying through a browser, you have to re-encapsulate the raw data into a know image file format, such as BMP, TGA, JPEG, PNG, etc. So you have to construct a file in binary mode (for BMP or TGA format this is pretty easy).

  • I actually would like to convert the raw bitmap data to base64 string so that it can be saved in the database before client display it in the browser. Do you know how to construct base64 string from the raw bitmap data? – cristie09 Nov 02 '17 at 18:51
  • The way you convert binary (octet stream) to base64 seem correct, there is an other example here : https://stackoverflow.com/questions/8305988/convert-binary-tostringencode64-back-to-binary So you can store the result in database. But as I said, if you want this to be displayed as image, once decoded again (in binary), you have to encapsulate pixel data in an image format the brower know. –  Nov 03 '17 at 09:51
  • Or you can reuse the raw pixel data in a texture in the client side... here, you have some choice depending what you want to do in the final stage. But browsers don't know what to do of an unidentified raw bytes buffer, you have to tell him, to tell him, you have to encapsulate into, for example, a BMP file. However, encapsulate pixel data in bitmap format is a low level binary operation, you'll find only examples in low level language such as C, you have to adapt using tools JS provides to you: https://en.wikipedia.org/wiki/BMP_file_format –  Nov 03 '17 at 10:05
  • ah ok, thank you for your explanation, it is helpful. Unfortunately although the conversion I did seem correct, you are right that i need to find a certain first before it is browser-readable. Simply converting the buffer to base64 isnt' useful. – cristie09 Nov 05 '17 at 10:09