2

I am currently combining two high resolution images into an html5 canvas element.

The first image is a JPEG and contains all color data, and the second image is an alpha mask PNG file.

I then combine the two to produce a single RGBA canvas.

The app is dealing with 2048x2048 resolution images, that need to also maintain their alpha channel. Therefore by using this method as ooposed to simply using PNG's, I have reduced the average filesize from around 2-3mb to 50-130kb plus a 10kb png alpha mask.

The method I use is as follows:

context.drawImage(alpha_mask_png, 0, 0, w, h);
context.globalCompositeOperation = 'source-in';
context.drawImage(main_image_jpeg, 0, 0, w, h);

Unfortunately this operation takes around 100-120ms. And is only carried out once for each pair of images as they are loaded. While this wouldn't normally be an issue, in this case an animation is being rendered to another main visible canvas (of which these high res images are the source art for) which suffers from a very noticable 100ms judder (mostly perceptible in firefox) whenever new source art is streamed in, loaded, and combined.

What I am looking for is a way to reduce this.

Here is what I have tried so far:

  • Implemented WebP for Google chrome, removing the need to combine the JPEG and PNG alpha mask altogether. Perfect in Chrome only but need a solution mostly for Firefox (IE 10/11 seems to perform the operation far quicker)
  • I have tried loading the images in a webworker and decoding them both, followed by combining them. All in pure javascript. This works but is far too slow to be of use.
  • I have also tried using WebP polyfills. Weppy is very fast and when ran in a webworker does not effect the main loop. However it does not support alpha transparency so is of no use which is a real shame as this method is very close. libwebpjs works okay within a webworker but like my manual decoding of the JPEG/PNG, is far too slow.

EDIT: To further clarify. I have tried transferring the data from my webworkers using transferrable objects and have even tried turning the result into a blob and creating an objectURL which can then be loaded by the main thread. Although there is no lag in the main thread any more, the images simply take far too long to decode.

This leaves me with WebGL. I have literally no understanding of how WebGL works other than I realise that I would need to load both the JPEG and PNG as seperate textures then combine them with a shader. But I really wouldn't know where to begin.

I have spent some time playing with the code from the following link:

Blend two canvases onto one with WebGL

But to no avail. And to be honest I am concerned that loading the images as textures might actually take longer than my original method anyway.

So to summarise, I am really looking for a way of speeding up the operation on high resolution images. (1920x1080 - 2048x2048) be it with the use of WebGL or indeed any other method.

Any help or suggestions would be greatly appreciated.

Community
  • 1
  • 1
gordyr
  • 6,078
  • 14
  • 65
  • 123
  • did you use transferable objects with your webworkers, or did you postMessage a bunch of string json data? – dandavis Sep 24 '13 at 16:45
  • @dandavis yes I did use transferrable objects. with both libwebpjs and the manual jpeg/png decoding it was mostly the decoding itself that took too long. A good suggestion though. I should have mentioned it. – gordyr Sep 24 '13 at 16:47
  • Just to be clear - are you combining these 2 images at animation time, or are you assembling and caching them before the animation begins? I'm thinking of pre-assembling the frames before animating. Whether I'd store the individual frames as ``s or `canvas`es or `imgData` objects would depend on speed, ease and memory use. – enhzflep Sep 24 '13 at 17:09
  • @enhzflep any downloaded images are being combined before hand. we currently precache 6 pairs. Then the animation begins. As the animation beings we download further pairs, it is when those ones are combined that we get we end up effecting the main loop. They are all cached as canvas elements. The trouble is, we can't download everything in advance since the dataset can theoretically be unlimited. For all intents and purposes, think of it as a constant stream of these image pairs. I really wish Mozilla would shape up and allow WebP support in Firefox :( – gordyr Sep 24 '13 at 17:14
  • The closest I have gotten is by storing these operations in an array which can then be called at any time. Since the animation depends on user interaction I constantly monitor that interaction, and fire off one of these combination calls whenever there is a lull. Although this does improve the perception somewhat it is far from perfect and can often lead to situations where the required source art isn't ready in time forcing the juddering function to execute immediately and making the stutter appear even worse etc. Still... no go.... – gordyr Sep 24 '13 at 17:24
  • @Ken-AbdiasSoftware I suppose you are right matey... It's a shame that the issue is really only noticable in Firefox. From further testing it appears that performance drops dramatically when using the 'source-in' globalComposite operation (2000 ops versus 200,000 ops) , when in other browsers it does not. I'll file this as a bug with Mozilla. I guess my options now are either further refinement of the "low user interaction detection" technique or to get my head around webp decoding and add alpha support to Weppy. Should I have any luck with the latter, all answer my own question here. – gordyr Sep 25 '13 at 08:34
  • When testing, did you use simple arrays or typed arrays (`Uint8Array`) ? – Sebastien C. Mar 24 '14 at 11:52
  • Is this a question you still need answered? I might have a solution but it might take a few hours to change from what i'm using it for to what you want it to do? If you in fact still need an answer i need to know which kind of png file you are using? (bit-depth), what is your primary channel for alpha? rgb black to white in a 24bit png, or the alpha channel of a 32bit png? – Snellface Oct 10 '14 at 10:59
  • I wonder if just drawImage the main image, then drawImage the Alpha, wouldn't be just right. (see this question and my answer here : http://stackoverflow.com/a/26202020/856501) – GameAlchemist Oct 22 '14 at 22:44

0 Answers0