7

I'd like to generate images in a bit of asm.js code running on a web worker. And I'd like to regularly composite the latest state of that computation onto a user-visible 2d canvas, together with some other content. Currently I have code which

  1. constructs an ImageData object using its constructor, based on a portion of the array buffer used by the asm.js code,
  2. calls createImageBitmap to turn the ImageData into an ImageBitmap,
  3. transfers that image bitmap from the worker to the GUI thread and
  4. uses that ImageBitmap as an argument to CanvasRenderingContext2D.drawImage.

Things work nicely in recent Chrome and Firefox, but Safari 9.1.3 apparently has no createImageBitmap function. How would I do something like the above in a way that works on Safari?

Is there some low-cost encoding of images, short of creating a data:image/png… for it? Is there some other way to turn a byte array into something you can feed to drawImage?

By the way: http://caniuse.com/ currently doesn't list this feature. There is a feature request which you can if you would like to see this feature monitored there.


If you prefer to see code for my current approach, here is the relevant portion of my worker:

var buffer = new ArrayBuffer(bufferSize);
var asm = Module.asm(self, {}, buffer);
var imgBytes = new Uint8ClampedArray(buffer, offset);
var imgData = new ImageData(imgBytes, width, height);
createImageBitmap(imgData).then(function(bmp) { // Not available on Safari!
    postMessage(bmp, [bmp]);
});

and here the corresponding GUI thread code:

var worker = new Worker(‹url of worker›);
worker.onmessage = function(msg) {
    var img = msg.data;
    context2d.drawImage(img, 0, 0, width, height);
};

The actual unabridged code is in this GitHub pull request, but there is a lot of other stuff which is irrelevant for the question at hand.

MvG
  • 57,380
  • 22
  • 148
  • 276

1 Answers1

5

Is there some other way to turn a byte array into something you can feed to drawImage?

You can post the ArrayBuffer of Uint8ClampedArray object to main thread; at main thread substitute using .putImageData() for .drawImage(). As indicated by @Kaiido, it is not necessary to create an ImageData object at Worker

var imgBytes = new Uint8ClampedArray(buffer, offset);
postMessage(imgBytes.buffer, [imgBytes.buffer]);

at main thread

worker.onmessage = function(e) {
  console.log(e.data); // `ArrayBuffer`
  ctx.putImageData(new ImageData(new Uint8ClampedArray(e.data), width, height), 0, 0);
}

http://plnkr.co/edit/N0v1YQHQX2rdFfHcOKeR?p=preview

guest271314
  • 1
  • 15
  • 104
  • 177
  • This loads the image from a PNG file. So while it is stored in a typed array, it's not raw pixel data. If I were to duplicate this approach, I'd likely have to PNG-encode the pixel data, which takes some time and requires either some libraries or some self-written code for PNG headers, row header insertion and minimal deflate encoding (e.g. using only the uncompressed store method). Still, it might be a viable option, particularly if nothing better turns up, so thanks! – MvG Oct 17 '16 at 21:30
  • @MvG You can alternatively transfer the `buffer` of `imgData` to main thread at `postMessage`, substitute using `.putImageData` for `.drawImage`; see updated post. – guest271314 Oct 20 '16 at 03:33
  • 1
    If I follow correctly, for your last part, you don't need to create an ImageData Object in the worker (https://jsfiddle.net/Kaiido/234ekx4b/) – Kaiido Oct 20 '16 at 04:46
  • @Kaiido Yes, you are correct. http://plnkr.co/edit/N0v1YQHQX2rdFfHcOKeR?p=preview\ – guest271314 Oct 20 '16 at 05:09
  • 1
    This sounds interesting. Since I need to composite with other content, I need `drawImage`, but I can `putImageData` to an off-screen canvas and then `drawImage` that to the primary. – MvG Oct 20 '16 at 06:14