67

You quite often read on the web that using closures is a massive source of memory leaks in JavaScript. Most of the times these articles refer to mixing script code and DOM events, where the script points to the DOM and vice-versa.

I understand that closures can be a problem there.

But what about Node.js? Here, we naturally don't have a DOM - so there is no chance to have memory leaking side effects as in browsers.

What other problems may there be with closures? Can anybody elaborate or point me to a good tutorial on this?

Please note that this question explicitly targets Node.js, and not the browser.

Golo Roden
  • 140,679
  • 96
  • 298
  • 425
  • I'm not sure I understand what you're looking for? You need to worry about how objects are managed, and a closure may make it less obvious an object is being held, but ... code could simply create too many objects and run out of memory. – WiredPrairie Oct 23 '13 at 19:46
  • Yes, of course. But are there any best practices, or are there patterns to watch out for to avoid memory leaks? Due to the asynchronous nature of Node.js I find it very difficult to imagine not to use closures. So, if I *have* to use them, what guidelines should I follow? Or, in other words, where may I run in trouble? – Golo Roden Oct 23 '13 at 19:48
  • 3
    Best guidance I could offer: "understand closures." Once you understand closures, it takes the mystery out of them, and you shouldn't need special guidelines for their use. – WiredPrairie Oct 23 '13 at 19:55
  • 2
    Well, I understand what closures are, but I'm lacking knowledge of memory management in V8, e.g. … hence I was asking for specific advice ;-) – Golo Roden Oct 23 '13 at 19:57

3 Answers3

47

This question asks about something similar. Basically, the idea is that if you use a closure in a callback, you should "unsubscribe" the callback when you are finished so the GC know that it can't be called again. This makes sense to me; if you have a closure just waiting around to be called, the GC will have a hard time knowing that you're finished with it. By manually removing the closure from the callback mechanism, it becomes unreferenced and available for collection.

Also, Mozilla has published a great article on finding memory leaks in Node.js code. I would assume that if you try out some of their strategies, you could find parts of your code that express leaky behavior. Best practices are nice and all, but I think it's more helpful to understand your program's needs and come up with some personalized best practices based on what you can empirically observe.

Here's a quick excerpt from the Mozilla article:

  • Jimb Esser’s node-mtrace, which uses the GCC mtrace utility to profile heap usage.
  • Dave Pacheco’s node-heap-dump takes a snapshot of the V8 heap and serializes the whole thing out in a huge JSON file. It includes tools to traverse and investigate the resulting snapshot in JavaScript.
  • Danny Coates’s v8-profiler and node-inspector provide Node bindings for the V8 profiler and a Node debugging interface using the WebKit Web Inspector.
  • Felix Gnass’s fork of the same that un-disables the retainers graph
  • Felix Geisendörfer’s Node Memory Leak Tutorial is a short and sweet explanation of how to use the v8-profiler and node-debugger, and is presently the state-of-the-art for most Node.js memory leak debugging.
  • Joyent’s SmartOS platform, which furnishes an arsenal of tools at your disposal for debugging Node.js memory leaks

The answers to this question basically say that you can help the GC out by assigning null to closure variables.

var closureVar = {};
doWork(function callback() {
  var data = closureVar.usefulData;
  // Do a bunch of work
  closureVar = null;
});

Any variables declared inside a function will go away when the function returns, except those that are used in other closures. In this example, closureVar has to be in memory until callback() is called, but who knows when that will happen? Once the callback has been called, you can give a hint to the GC by setting your closure variable to null.

DISCLAIMER: As you can see from the comments below, there are some SO users who say that this information is out of date and inconsequential for Node.js. I don't have a definitive answer on that yet; I'm just posting what I've found on the web.

Community
  • 1
  • 1
RustyTheBoyRobot
  • 5,891
  • 4
  • 36
  • 55
  • I agree with plalx, setting null does nothing. That was a trick necessary in older IE browsers because of bad garbage collection. There is hardly any evidence that it causes problems for V8 or Mozilla Seamonkey. The question you link is two years old too, I doubt it would be relevant here. In the same question delnan's comment says circular closures not to be a problem for V8. – user568109 Oct 31 '13 at 03:30
  • 1
    I too read the question and thought of answering as closures being unsafe for collection. But on validating this on net, I found all such articles relate to DOM, hinting browser related GC problems (IE ?). That too several years old. I could not find any evidence stating such patterns in closure were failed to be collected by V8 GC. Moreover there is no benchmark to prove this technique works for V8. – user568109 Oct 31 '13 at 03:40
  • I haven't had time to run test code on the "set variables to null" strategy, so I added a disclaimer. Do either of you (@plalx or @user568109) have a reference to someone's benchmarks that disproves that SO answer? – RustyTheBoyRobot Oct 31 '13 at 15:18
  • 1
    @user568109 I must point out that plax's comment is valid in the context of an event listener. But saying here that _setting null does nothing_ will probably make some readers think that this technique is not helpful in general, which is wrong. What really counts is that the variable points to allocated memory. If it is reachable, it won't be free'd. When you null the variable the allocated memory is not reachable and will be garbage collected. – borisdiakur Mar 02 '14 at 20:32
  • @Lego You did the whole comment right and do note that this question is tagged node.js and OP specifically asks for node.js. I am not talking about the general scenario. The OP asks how it is different from the usual browser scenario. Also I cannot stress this enough about not posting old blogs and heresay. There is hardly any real evidence it will work for node. It may have been a valid back then now it is not. – user568109 Mar 03 '14 at 05:25
  • @user568109 Any reference? I've node v0.10.24 installed and still run into and resolve issues with memory leaks using the technique above. – borisdiakur Mar 03 '14 at 08:33
14

You can find a good example and explanation in this blog post by David Glasser.

Well, here it is (I added a few comments):

var theThing = null;
var cnt = 0; // helps us to differentiate the leaked objects in the debugger
var replaceThing = function () {
    var originalThing = theThing;
    var unused = function () {
        if (originalThing) // originalThing is used in the closure and hence ends up in the lexical environment shared by all closures in that scope
            console.log("hi");
    };
    // originalThing = null; // <- nulling originalThing here tells V8 gc to collect it 
    theThing = {
        longStr: (++cnt) + '_' + (new Array(1000000).join('*')),
        someMethod: function () { // if not nulled, original thing is now attached to someMethod -> <function scope> -> Closure
            console.log(someMessage);
        }
    };
};
setInterval(replaceThing, 1000);

Please try it out with and without nulling originalThing in Chrome Dev Tools (timeline tab, memory view, click record). Note that the example above applies to browser and Node.js environments.

Credit also and especially to Vyacheslav Egorov.

eden
  • 105
  • 2
  • 9
borisdiakur
  • 10,387
  • 7
  • 68
  • 100
  • Can you add the other examples mentioned. Also mention that this snippet is expected to leak and you can plug it with extra line setting it to null. – user568109 Mar 04 '14 at 01:52
  • @borisdiakur Thanks for that snippet.. after a long review evening it was the first really helpful post! But please let me ask one more question... Whats the suitable way to handle this behavior? Setting each variable null in any callback which is not shared? Actually I run node with --expose-gc and call periodically global.gc() and it keeps memory stable compared to a second instance without (100Mb vs. continuos growing 300Mb) – Bernhard Sep 19 '15 at 00:24
  • @Bernhard Running with `--expose-gc` and calling `gc()` periodically is bad practice. V8 collects garbage as needed via [incremental marking and lazy sweeping](http://jayconrod.com/posts/55/a-tour-of-v8-garbage-collection). A suitable way to handle the behavior above is to null the local variabale which points to the object that has become a part of the lexical environment. – borisdiakur Sep 21 '15 at 18:22
  • @borisdiakur Thanks thats correct and I've found similar information about that. But how does node handle the memory internally because even after a long time the memory isn't decreasing when I'm looking via htop. Or is the application just keeping the memory instead of returning free memory to operating system -> because a heapdump size is only 25% of the memory which is in 'use' – Bernhard Sep 22 '15 at 09:14
  • 1
    @Bernhard I recommend reading [A tour of V8: Garbage Collection](http://jayconrod.com/posts/55/a-tour-of-v8-garbage-collection). My experience (with an application that consumes a lot of memory) is that V8 frees memory only as soon as there is not enough memory left. For instance if you run your app with `--max-old-space-size=3072` and do not consume more than 3GB then it doesn't free memory at all. – borisdiakur Sep 23 '15 at 20:49
  • @borisdiakur Thanks for the explanatino with this great example, please allow me to ask one more question, I made a little bit changes with the snippet by removing the someMethod function in the theThing object and seems like memory leak issue is fixed, may I ask why? I thought originalThing is still referenced by unused function and should not be garbage collected. – chen.w Sep 16 '21 at 14:58
  • @chen.w The memory leak goes away when you remove `someMethod`, because that was the only closure that was "still accessible" from the global object/tree (through the `theThing` variable). Since that inner closure is no longer being added to `theThing`, the entire `replaceThing` scope is able to be released as soon as each execution of `replaceThing` completes. (no expert, but I'm pretty sure this is why) – Venryx Jan 13 '22 at 21:51
2

I have to disagree with closures being a cause of memory leaks. This maybe true for older versions of IE because of its shoddy garbage collection. Please read this article from Douglas Crockford, which clearly states what a memory leak is.

The memory that is not reclaimed is said to have leaked.

Leaks are not the problem, efficient garbage collection is. Leaks can happen in both browser and server JavaScript applications alike. Take V8 as an example. In the browser, garbage collection happens on a tab when you switch to different window/tab. The leak is plugged when idle. Tabs can be idle.

On the server things are not so easy. Leaks can happen, but GC is not as cost-effective. Servers cannot afford to GC frequently or its performance will be affected. When a node process reaches a certain memory usage, it kicks in the GC. Leaks will then be removed periodically. But leaks still can happen at a faster rate, causing programs to crash.

RustyTheBoyRobot
  • 5,891
  • 4
  • 36
  • 55
user568109
  • 47,225
  • 17
  • 99
  • 123
  • Any constructive criticism is welcome. The downvoter please add the comments. – user568109 Oct 30 '13 at 14:22
  • 1
    -1 I guess some readers might interpret your answer as _closures cannot be the cause of memory leaks_. Memory leaks can hide very well in closures: https://github.com/jedp/node-memwatch-demo#some-examples-of-leaks – borisdiakur Mar 02 '14 at 20:41
  • @Lego I cannot stress this enough. Read the answer completely. It seems you are just reading the first line. If you do not have a good understanding of the matter at hand, please refrain from downvoting. Your point is that memory leaks can happen because of closures. I don't deny it. Mine is V8 has a fairly good garbage collection that can handle leaks that arise from closures. Your link itself proves my point. The garbage collection takes a toll when they happen slowly over time(asynchronously) or at a rapid pace(lots of leaks to plug). – user568109 Mar 03 '14 at 05:42
  • 3
    Please look at the third example listed in [this blog post](https://www.meteor.com/blog/2013/08/13/an-interesting-kind-of-javascript-memory-leak) and try it in Chrome Dev Tools (timeline tab, memory view, click record). You can click on the garbage can icon as much as you like, v8 gc won't collect the allocated memory. Seems to me that v8 gc is not as smart enough as you claim. I'll be happy to get a better understanding and also to remove my downvote if you prove me wrong (I don't like to downvote, it makes me loose one internet point, and I always read the full answer before downvoting) : ] – borisdiakur Mar 03 '14 at 08:08
  • @Lego If you read the following [discussion](https://news.ycombinator.com/item?id=5959020) and the [original post](http://point.davidglasser.net/2013/06/27/surprising-javascript-memory-leak.html). This is expected behaviour. An object is not elligible for garbage collection as long as one closure is still referencing. The downside for this is that any other closures which are not referencing it and want to be GCed are hung up. For some this is bad and they want it to be reclaimed (even with referencing closure there because they cannot control it). That is beyond V8's control. – user568109 Mar 03 '14 at 11:21
  • @Lego Although the scenario maybe leaky because you are not giving it a chance to GC :P. But setting object as null does make the other hanging closures eligible for GC! This seems to be different than the "setting an obj as null" in the previous answer. All examples mentioned there refer to old posts relating to IE. The blog post you mention has better examples than that. If you are willing to post it as an answer I will be happy to upvote. – user568109 Mar 03 '14 at 12:02
  • I know this is a trap and you just want to downvote me back, but ok, I've posted it as an answer ; ] – borisdiakur Mar 03 '14 at 12:37
  • 1
    @borisdiakur this example involves global variable, which leads to root cause of the leak - global variables. The closure have nothing to with the leak. But yes, they can *hide* leaks. – setec Aug 22 '16 at 11:07
  • 1
    This is turning into a semantic argument but I think the most salient thing to know is that closures can easily *exacerbate* memory leaks initially caused by global variables, event listeners that were not cleaned up, etc. Whatever memory the closure internally refers to *is leaked* even if the closure isn't the root cause of the leak. – Andy Mar 13 '19 at 07:29