1

I need some help with finding a memory leak in a small, Browser / WebWorker JavaScript. I tracked it down into this little piece of code:

    /**
     * Resizes an Image
     *
     * @function scaleImage
     * @param {object}  oImageBlob  blob of image
     * @param {int}     iHeight     New height of Image
     * @return {ImageBitmap}    ImageBitmap Object
     */         
    async function scaleImage(oImageBlob, iHeight) {
        var img = await self.createImageBitmap(oImageBlob); 
        var iWidth = Math.round( ( img.width / img.height ) * iHeight); 
        var canvas = new OffscreenCanvas(iWidth,iHeight);
        var ctx = canvas.getContext('2d');  
        ctx.drawImage(img, 0, 0, canvas.width, canvas.height);      
        return(canvas.transferToImageBitmap());
    }

It's called from:

    [inside a web worker: Some looping that calls this about 1200 times while parsind files from a s3 bucket ...]
         var oImageBlob = await new Response(oS3Object.Body, {}).blob();
         var oThumbnail = await scaleImage(oImageBlob, 100);
         await IDBputData(oInput.sFileKey, oImageBlob, oInput.sStore, oThumbnail)
    [... end of the loop]

The other interior function is

    /**
     * Put information to IndexedDB 
     *
     * @function IDBputData 
     * @param {string}  sKey        key of information
     * @param {string}  sValue      information to upload
     * @param {string}  sStore      name of object store
     * @param {object}  oThumbnail  optrional, default: null, thumbnail image
     * @return {object}     - SUCCESS: array, IndexedDB Identifyer "key"
     *                      - FAIL: Error Message
     */     
    async function IDBputData(sKey, sValue, sStore, oThumbnail=null) {
        var oGeneratedKeys = {};
        if(sStore=="panelStore"){
            oGeneratedKeys = await getKeyfromSKey(sKey);
        }
        return new Promise((resolve, reject) => {
            const tx = oConn.transaction(sStore, 'readwrite');                  
            const store = tx.objectStore(sStore);
            var request = {}
            request = store.put({panelkey: oGeneratedKeys.panelkey, layerkey: oGeneratedKeys.layerkey, countrycode: oGeneratedKeys.countrycode, value: sValue, LastModified: new Date(), oThumbnail: oThumbnail});
            request.onsuccess = () => (oThumbnail.close(),resolve(request.result));
            request.onerror = () => (oThumbnail.close(),reject(request.error));
        });
    }

When I run it this way in Chrome, it will consume every bit of RAM I've got free (around 8 GB) and then crash. (Laptop with shared RAM for CPU/GPU).

When I change

         var oThumbnail = await scaleImage(oImageBlob, 100);

to

         var oThumbnail = null;

RAM consumption of Chrome stays rather fixed around 800 MB, so there must be something with the topmost function "scaleImage".

I tried tweaking it, but with no success.

/**
 * Resizes an Image
 *
 * @function scaleImage
 * @param {object}  oImageBlob  blob of image
 * @param {int}     iHeight     New height of Image
 * @return {ImageBitmap}    ImageBitmap Object
 */         
async function scaleImage(oImageBlob, iHeight) {
    var img = await self.createImageBitmap(oImageBlob); 
    var iWidth = Math.round( ( img.width / img.height ) * iHeight); 
    var canvas = new OffscreenCanvas(iWidth,iHeight);
    var ctx = canvas.getContext('2d');  
    ctx.drawImage(img, 0, 0, canvas.width, canvas.height);      

    var oImageBitmap = canvas.transferToImageBitmap();

    ctx = null;
    canvas = null;
    iWidth = null;
    img = null;

    return(oImageBitmap);
}

Any help is very much appreciated.

Kaiido
  • 123,334
  • 13
  • 219
  • 285
HOK
  • 225
  • 1
  • 11
  • 1
    When the request created with `store.put` resolves you should clean up the image bitmap. Example `request.onsuccess = () => (oThumbnail.close(), resolve(request.result));` (do same in `onerror`) Calling `ImageBitmap.close()` disposes of resources used by the bitmap. – Blindman67 Apr 18 '21 at 12:48
  • @Blindman67 That sounded great, but after testing it a few minutes ago, I can say: it didn't make any difference. :-( I updated the code in the question accordingly. But there must be something else I am missing. :-( – HOK Apr 18 '21 at 14:25
  • 1
    Do you ever clean up those offscreen canvases you constantly create `var canvas = new OffscreenCanvas(iWidth,iHeight);`? – obscure Apr 18 '21 at 14:42
  • @obscure According to my research canvas = null; should do the trick. I tried that in the last code snippet of the question. You think I did something wrong, there? – HOK Apr 18 '21 at 14:55
  • @obscure The canvas is scoped to the function and is thus de-referenced when the function return. – Blindman67 Apr 18 '21 at 14:55
  • @Blindman67 - I'm not exactly sure - that might depend on what's actually happening in the constructor of OffscreenCanvas so it might not be enough to null the reference just in the calling function. – obscure Apr 18 '21 at 15:00
  • @Heinz I'm not sure - it can't hurt to post the class definition of OffsceenCanvas though. – obscure Apr 18 '21 at 15:00
  • Why are you passing the blob to upload. That is the only other likely source of a memory not being released however the code does not show what happens to the blob – Blindman67 Apr 18 '21 at 15:04
  • @obscure offscreenCanvas is a standard Javascript class for web workers – HOK Apr 18 '21 at 15:08
  • @obscure `canvas.transferToImageBitmap()` does not hold the canvas – Blindman67 Apr 18 '21 at 15:08
  • @blindman the blob becomes sValue. It's saved in the IndexedDB by the put statement (so there's both, the original and the thumbnail in the IndexedDB – HOK Apr 18 '21 at 15:09
  • Without more code we can only guess. Are you sure `scaleImage` is resolving? Have you traced the execution to ensure either onsuccess and onerror get called BEFORE you start the next entry? Even so 1200 100*100 px thumbs is only ~50Mb and the Blob to use up 8Gb would have to be ~6.5Mb each. I cant remember the DB size limit but surly you are not trying to add 8Gb+ of data to the store? – Blindman67 Apr 18 '21 at 16:46
  • Got it! It's not ```img = null;``` but ```img.close();``` My wild guess: img = null leaves it to the garbage collector, which is meant for RAM (CPU). The ImageBitmap object was designed for VRAM (GPU) [what on my development laptop is physically the same]. Therefore it is important to use the correct destructor "close()". The two of you were utterly helpful. If any of you want to author the answer, I'd happily accept it to create some minor merits. ;-) – HOK Apr 18 '21 at 20:28

1 Answers1

3

For the ImageBitmap to release its bitmap data the most efficient way, you have to call its .close() method once you're done with it.

But actually, you don't need this scaleImage function. createImageBitmap() has a resizeHeight option, and if you use it without the resizeWidth one, you'll resize your image by keeping the aspect-ratio exacty like you are doing in your function, except that it won't need to assign the bitmap twice.

Once you have this resized ImageBitmap, you can transfer it to a BitmapRenderingContext (which will internally close() the original ImageBitmap) and call the transferToBlob() from that renderer. This should be lighter for your computer.

async function worker_script() {
  const blob = await fetch( "https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png" ).then( (resp) => resp.blob() );
  // resize from createImageBitmap directly
  const img = await createImageBitmap( blob , { resizeHeight: 100 } );
  const canvas = new OffscreenCanvas( img.width, img.height );
  canvas.getContext( "bitmaprenderer" )
    .transferFromImageBitmap( img ); // this internally closes the ImageBitmap
  const resized_blob = await canvas.convertToBlob();
  // putInIDB( resized_blob );
  // for the demo we pass that Blob to main, but one shouldn't have to do that
  // show the canvas instead ;)
  postMessage( { blob: resized_blob, width: img.width } );
  // to be safe, we can even resize our canvas to 0x0 to free its bitmap entirely
  canvas.width = canvas.height = 0;
}

// back to main
const worker = new Worker( getWorkerURL() );
worker.onmessage = ({ data: { blob, width } }) => {
  const img = new Image();
  img.src = URL.createObjectURL( blob );
  img.onload = (evt) => URL.revokeObjectURL( img.src );
  document.body.append( img );
  console.log( "ImageBitmap got detached?", width === 0 );
};

function getWorkerURL() {
  const content = "!" + worker_script.toString() + "();";
  const blob = new Blob( [ content ], { type: "text/javascript" } );
  return URL.createObjectURL( blob );
}
Kaiido
  • 123,334
  • 13
  • 219
  • 285
  • Thanks. I agree with your solution, however, I have got two remarks: -> createImageBitmap() and resizeHeight work. But, even with quality "high", it results in a noticeably lower quality. Also my test with a few hundred Full-HD files showed it's as fast as or even slower in 80% of the cases on Chrome. (No, I wouldn't have guessed that, either. :-) ) -> Blindman67 and obscure were first in helping me find the right solution (see comments of the question), so I'll wait another day if one of them wants to post an answer to be fair. – HOK Apr 19 '21 at 06:05
  • Your tests results are really weird... There is definitely less being done going through this path (I spent last month in Chrome's implementation fixing an unrelated bug there). If I find the time I'll setup a test myself to see what's going on, but for a start, can you double check all Hardware Acceleration is enabled on your system? (chrome://gpu/). And reading the thread of comments there, only Blindman67 gave you correct advice. – Kaiido Apr 19 '21 at 06:19
  • If it helps, I can try to share details about my test setup. Everything under "Graphics Feature Status" is enabled or Hardware accelerated, despite "Vulkan" My impression was, that not passing the optional params to createImageBitmap might skip a bit of the code that works slower than the canvas constructor. I don't know the implementation side of chorme, though it's just a guess. – HOK Apr 19 '21 at 06:24
  • Running a really quick test (https://jsfiddle.net/dn9mofur/2/ ), on my computer the bitmaprender outperforms the 2D context by a lot: 15s vs 21s for 100 images with full copy of the original Blob ([avoiding this actually useless task](https://jsfiddle.net/dn9mofur/) I end up with 13s vs 19s). – Kaiido Apr 19 '21 at 06:58
  • And regarding quality there indeed seem to be something off when downscaling with resizeQuality "high" in createImageBitmap... There is a bigger difference between "low" and "medium" than between "low" and "high"... And it seems drawImage draws the same with "high" and "medium" oO – Kaiido Apr 19 '21 at 08:21
  • OK, I'll give it a shot with "medium",later today. :-D – HOK Apr 19 '21 at 09:52
  • Actually the type of the source does change both drawImage's and createImageBitmap's sampling... I'll havr to write more tests to see what happens, but yes, from eyes, downsampling a Blob by createImageBitmap looks better with "medium"... – Kaiido Apr 19 '21 at 10:02
  • I narrowed it down. It widely depends on the images I feed it with. jpgs that already have a rather high compression tend to favour the "canvas" solution whereas high quality/low compressed give the "createImageBitmap" resize an edge. In 6 groups selected according compression (with overall 1200 files) I experienced createImageBitmap between 7% and 36% faster. At the high compressed (7% and 8% faster at a bigger sum of images) the measurement was very volatile. So I doubt the cases where canvas was faster is due to canvas, but due to external factors as both are incredibly fast. – HOK Apr 19 '21 at 12:21
  • Same is for Quality. The visible difference between createImageBitmap "medium" and "scaleImage" is high, when the base file already shows artifacts due to high compression. It looks more or less identical, when the source image quality is high. I'm still comparing scale image medium and high, but for the moment it looks like tehre's something wrong, as "high" is always faster and often uglier. – HOK Apr 19 '21 at 12:25