1

I have a little piece of code as such :

async function saveIntoMp4(chunks) {
    const options = {
        type: "video/webm"
    }
    let blob = new Blob(chunks, options);
    chunks.length = 0 // to stop any memory leaks
    const buffer = Buffer.from(await blob.arrayBuffer());
    try {
        fs.writeFile(
            `./videos/1.mp4`,
            buffer,
            () => console.log("video is saved!")
        );
    } catch (error) {
        console.log(error)
    }
}

And everytime this function is called the occupied memory of mine goes up. It reached 4gb in fact. I have tracked the heap usage with node --inspect and I realized that arrayBuffer is infact emptied out without an issue. Yet the memory that started at 75mb reached more than 100mb just by calling the function once as is shown in the picture. Is this a memory leak? How can I stop this from happening. I have tested it on Ubuntu 20 / 22 / Centos 9 and Rocky linux ( red hat variant ) all the same situation. The only place I am not facing memory issues is my windows 11. :

enter image description here

Hypothesis
  • 1,208
  • 3
  • 17
  • 43
  • *"How can I stop this from happening."* - stop worrying about it? nodejs garbage collection will do its job when it needs to or [call it manually](https://stackoverflow.com/a/30654451). though, i do curious your nodejs version - some build might have bugs or regression. – Bagus Tesa Jun 26 '23 at 11:40
  • @BagusTesa I called it manually as well by calling global.gc() in an interval of a minute. It went up to 4gb at one point. The server rebooted itself. It just does not work. – Hypothesis Jun 26 '23 at 11:42
  • wait, server? is it on idle or swarmed by request? often times its not programming mistake but someone ran ddos against your server. – Bagus Tesa Jun 26 '23 at 11:47
  • @BagusTesa No, server is only accessible from my ip and I am the only one using it. I am pretty sure of that. There is some leakage that I cannot figure out. – Hypothesis Jun 26 '23 at 11:53
  • may we know what this `chunks` is? – Bagus Tesa Jun 26 '23 at 12:24

1 Answers1

0

Okay so this had been bothering me for 10 days, I believe this had something to do with the memory allocator according to https://github.com/Unitech/pm2/issues/4375

I installed libjemalloc through apt-get on Ubuntu 20 by

echo "/usr/lib/x86_64-linux-gnu/libjemalloc.so.2" >> /etc/ld.so.preload

and checked using

ps aux | grep node cat /proc//smaps | grep jemalloc

The problem seems to be fixed.

Hypothesis
  • 1,208
  • 3
  • 17
  • 43