101

At startup, it seems my node.js app uses around 200MB of memory. If I leave it alone for a while, it shrinks to around 9MB.

Is it possible from within the app to:

  1. Check how much memory the app is using ?
  2. Request the garbage collector to run ?

The reason I ask is, I load a number of files from disk, which are processed temporarily. This probably causes the memory usage to spike. But I don't want to load more files until the GC runs, otherwise there is the risk that I will run out of memory.

Any suggestions ?

Rahul Iyer
  • 19,924
  • 21
  • 96
  • 190
  • How are you loading these files? If you need to load a huge file, you must use the Stream approach and you can process the file while still reading. Have you consider it? – well Oct 05 '17 at 12:39
  • @well That was 3 years ago. I'm working on some other project now and can't remember much about it. – Rahul Iyer Oct 08 '17 at 04:12

3 Answers3

178

If you launch the node process with the --expose-gc flag, you can then call global.gc() to force node to run garbage collection. Keep in mind that all other execution within your node app is paused until GC completes, so don't use it too often or it will affect performance.

You might want to include a check when making GC calls from within your code so things don't go bad if node was run without the flag:

try {
  if (global.gc) {global.gc();}
} catch (e) {
  console.log("`node --expose-gc index.js`");
  process.exit();
}
evandrix
  • 6,041
  • 4
  • 27
  • 38
IgnisFatuus
  • 1,998
  • 1
  • 12
  • 8
  • 2
    Where the docs for "--expose-gc"? I am not find any docs in nodejs.org. – pea3nut Jun 27 '19 at 12:31
  • 15
    @pea3nut it's a option of the V8 runtime, which node uses. This and other options can be listed with `node --v8-options` – janispritzkau Oct 03 '19 at 15:38
  • 5
    You have to pass true to the function for full GC otherwise it will just do a minor GC – Piyush Balapure Oct 20 '19 at 10:54
  • @PiyushBalapure, do you have any reference about gc() arguments ? – Franck Freiburger Nov 12 '19 at 15:30
  • I am yet to see the documentation for the gc() arguments. Did not even find it in V8 docs. I read about full GC here https://nodesource.com/blog/request-garbage-collection-for-node-js-processes-with-n-solid – Piyush Balapure Nov 13 '19 at 14:40
  • 4
    FWIW, passing `true` (these days, at least) effects a _minor_ collection: https://github.com/nodejs/node/blob/5421e15bdc6122420d90ec462c3622b57d59c9d6/deps/v8/src/extensions/gc-extension.cc#L65-L71 – cartant Feb 22 '21 at 08:02
  • 5
    When would the catch block run? If `--expose-gc` is missing then `global.gc` will just be undefined – rob Mar 05 '21 at 02:41
  • 1
    @evandrix edited this answer and Pique's answer. They were seemingly trying to combine them (why??) and ended up making them identically broken. Those edits need to be reverted. – forresthopkinsa Jul 09 '21 at 22:20
  • "Keep in mind that all other execution within your node app is paused until GC completes" -- what other execution are you referring to (apart from worker threads)? Do you simply mean this method is blocking (and takes a while to return) and as-such, the event loop won't be processing other callbacks until it completes? – Lawrence Wagerfield Oct 02 '21 at 14:23
  • 1
    Guys it's 2022, `global?.gc()` suffices – Ali Mert Çakar Sep 15 '22 at 18:53
  • What’s the point of doing `try`–`catch`ing when you’re already checking `if (global.gc)`? – Константин Ван Nov 29 '22 at 19:28
5

When you cannot pass the --expose-gc flag to your node process on start for any reason, you may try this:

import { setFlagsFromString } from 'v8';
import { runInNewContext } from 'vm';

setFlagsFromString('--expose_gc');
const gc = runInNewContext('gc'); // nocommit
gc();

Notes:

This method should be used with care. Changing settings after the VM has started may result in unpredictable behavior, including crashes and data loss; or it may simply do nothing.

TmTron
  • 17,012
  • 10
  • 94
  • 142
-9

One thing I would suggest, is that unless you need those files right at startup, try to load only when you need them.

EDIT: Refer to the post above.

lwang135
  • 576
  • 1
  • 3
  • 13
  • I need the files at startup. And actually, once the GC runs, my app uses only 9MB of memory for the lifetime of its existence. All I do is require a JSON object (that is a dictionary). Since this code is internal to node.js I can't optimise it. But I need the dictionary to be available for search suggestions. – Rahul Iyer Dec 05 '14 at 19:01
  • I'm a bit confused. How does it go from `almost no memory` down to 9MB if you need this object for search suggestions? I assume those should remain for the lifetime for your app.. – lwang135 Dec 05 '14 at 19:02
  • what do you mean by almost no memory ? Yes I "require" the json file doing something like var foo = require('./mydictionary'), and it exists for the lifetime of my app. At startup the memory usage of my node.js app goes up to 200MB! (I don't understand why you say almost no memory), and then after some time goes down to 9MB, even though the dictionary is still in memory. – Rahul Iyer Dec 05 '14 at 19:06
  • Sorry I asaid "almost no memory" because you said " But I don't want to load more files until the GC runs, otherwise there is the risk that I will run out of memory." Perhaps the right thing to do, is to look for what makes your app go to 200MB. If the dictionary is in the memory, that shouldn't be the cause of it. – lwang135 Dec 05 '14 at 19:59
  • Its just a require statement require('./foo'). JSON object with lists of strings. My server has just 1GB of memory, so I'm just playing it safe. – Rahul Iyer Dec 05 '14 at 20:41
  • `I don't think you can call the Garbage Collector directly` that is not true. You can as the selected answer suggested. – TBE Apr 22 '16 at 11:47