13

I was running my server with pm2 start ... and pm2 monit was showing me 3GB memory after 2 hours. So I attached memwatch, now I waited for another 2 hours and again the memory shown by pm2 monit reached 3GB.

So, I checked the logs memwatch generated. Showed me:

{ before: { nodes: 75659, size_bytes: 11141702, size: '10.63 mb' },
  after: { nodes: 73226, size_bytes: 10840598, size: '10.34 mb' },
  change:
   { size_bytes: -301104,
     size: '-294.05 kb',
     freed_nodes: 5141,
     allocated_nodes: 2708,

And another:

{ before: { nodes: 72591, size_bytes: 10728318, size: '10.23 mb' },
  after: { nodes: 73284, size_bytes: 10798062, size: '10.3 mb' },
  change:
   { size_bytes: 69744,
     size: '68.11 kb',
     freed_nodes: 5931,
     allocated_nodes: 6620,

Now I am really confused, those are the last logs so I'm pretty sure those are the logs generated when pm2 monit showed the huge memory leak.

So, why is memwatch showing me 10MB+ memory and pm2 monit showing 3GB+?


Now switching to something like forever or monit to see if the leak still exists.


A bit more background

  • I have been trying to profile and find the leak and there isn't just any leak showing on the profiles.
  • The memwatch diff is started when a client connects and the difference is taken when the client disconnects.
majidarif
  • 18,694
  • 16
  • 88
  • 133
  • About this, I was trying to profile it for the last 2 days and just can't find the problem. So I finally decided to move-on to Golang, which is amazing, huge memory and performance difference. Very efficient. Finally no confusing leaks. Plus its pretty easy to code in, recoded javascript to golang in less than a day. – majidarif Feb 17 '15 at 16:09

2 Answers2

7

There are a lot of really bad answers on this thread. Really, really bad.

The answer is simple: your code has a memory leak that you need to find and remove, and it's VERY UNLIKELY that the GC is wrong, it's far more likely your code is the culprit.

So first off:

  1. "I found that with PM2 the garbage collector doesn't work" - this is patently nonsensical, PM2 runs node, which uses V8, which has its own garbage collector built-in. This isn't disabled just because you run PM2.
  2. "So until that is fixed call the GC" - No! Calling the GC manually is lazy if your code has leaks, you should fix them.
  3. "After removing pm2 there is no more memory leak for now." - No, you're simply not monitoring it in PM2; the leak will still be there, you just don't see the error because you're running it using node and not looking at the results.

Secondly, the actual solution should be:

Profile your code's memory and CPU usage using the tool of your choice. For most people, that is going to be Chrome's debugging tools connected to the running node instance, and look at which functions are causing memory usage to spike. Simply calling the GC manually is lazy because your code still has a leak but you're just telling V8 to constantly hoover up memory after your app leaks it, which is very lazy practice.

user-12410035
  • 277
  • 4
  • 2
  • 2
    super simplified code proved that the code wasn't leaking since it didn't leak when executed without pm2. the question is 5 years old, it must have been fixed now. – majidarif Dec 04 '19 at 15:51
  • There was no super simplified code provided. The example by Vijay above is not the simplified code, it's the code to force garbage collection. The issue was resolved because the GC was spammed, but that just masks the issue rather than resolve it. – user-12410035 Dec 05 '19 at 17:48
  • The reason I posted that is because we had someone use this answer and this absolutely spammed the garbage collector and caused huge CPU usage because the GC was being told to recycle many times per second. – user-12410035 Dec 05 '19 at 17:49
  • 4
    I can attest to this. I inspected my application in dev tools and found that I had no leaks. Left it for hours, still no leaks. I ran it in pm2, and the memory kept going up slowly. Then, I had set all memory hoarding variables and functions to null, ran node (with the exception of the imports) so that otherwise it should be hoarding 0 bytes. In dev tools, it was okay. in pm2? Voila! leak. Something's definitely up with pm2 – user2993497 Dec 20 '19 at 15:53
  • I've had the exact opposite. With code that leaks in node, it leaks, and in pm2 it leaks. Fix the leaks in both, no leaks in both. This could be with some old version of pm2, but latest it certainly does not do this. Leaky code in either is the cause of this. – user-12410035 Jan 13 '20 at 15:35
  • 2
    I'm observing the same behaviour as the other guys - when i start the script using just Node, it keeps around 30-50MB of memory. When I start it using PM2, PM2 reports 70MB initially and then it grows slowly. I don't know if it's a problem with PM2 or the way that i am using it, but it doesn't look like a problem with the script that's being run. Oh, and the memory profiler doesn't show any change - even if i run PM2, when I attach the profiler to the child process, it shows 35-40MB while PM2 shows 240. – Milen Georgiev Jan 10 '21 at 04:24
  • 1
    https://stackoverflow.com/questions/59474575/pm2-incorrect-memory-usage-reading-possible-memory-leak-with-node-js-applica?noredirect=1&lq=1 - newer post on SO that suggest a problem with PM2 – Milen Georgiev Jan 10 '21 at 04:36
3
  • I was also facing the same issue, but after little research i found that nodejs is not calling the garbage collector when using pm2.
  • So, until PM2 fixes that issue temporary work around is force call garbage collector, using following

pm2 start app.js --node-args='--expose-gc'

Above argument --expose-gc will allow us to force call garbage collector from node js, now use following code for force garage collection.

if (global.gc) {
   global.gc();
} else {
   console.log('Garbage collection unavailable.  use --expose-gc '
   + 'when launching node to enable forced garbage collection.');
}  

This will solve PM2 meomry leak problem.

Vinay K
  • 463
  • 6
  • 18
  • 14
    Can you provide the source for `but after little research i found that nodejs is not calling the garbage collector when using pm2.`. Thanks. – majidarif Dec 04 '15 at 16:43
  • 1
    Memory leak happened for us too with pm2 in Node Js. After removing pm2 there is no more memory leak for now. – Ashutosh Chamoli May 17 '19 at 04:55