15

I need to update a lot of data within a given interval with JavaScript. The problem is, no matter of what JS library i use (even bare-bone js), that all browsers seem to allocate memory on every AJAX request and are not able to free it afterwards. Here is a sample snipped that should reproduce the error:

    <!DOCTYPE html>
    <html lang="en">
        <head>
            <title>Memleak Test</title>
            <meta charset="utf-8" />
            <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.5.2/jquery.min.js"></script>
            <script type="text/javascript">

                function readData() {
                    $.getJSON('data.php');
                }

                $(document).ready(function() {
                    setInterval(readData, 1000);
                });
            </script>
        </head>
        <body>
            <div id="content"></div>
        </body>
    </html>

An equivalent test page is available at jsbin

Here is more info on this:

  • I also tried to put the readData() function as a closure directly in the setInterval() call. This doesn't seem to make any difference.
  • I use jQuery here, but any other Library would produce the same errors.
  • My data.php script just produces a fake JSON-Object with json_encode() in PHP.
  • I know that one second is a short timeframe here, in my production script the timeframe is 30 seconds. I just wanted to see the effect quicker (in the production app it takes hours but then the memory is full too).
  • The problem here is that the app will be open 24/7.

It seems so simple that I think I'm doing something really wrong here, it would be great if some of the JS gurus in here can help me out!

UpTheCreek
  • 31,444
  • 34
  • 152
  • 221
daschl
  • 1,124
  • 6
  • 11
  • 1
    In your *real* test are you manipulating the DOM, via `.html` or otherwise? – karim79 Apr 18 '11 at 13:05
  • karim79, in my production environment of course i manipulate the DOM. When i tried to track down the problem, I could narrow it down to the ajax call and to my surprise this "leak" also happens without any DOM manipulation. I suppose the browser is not able to free up the memory allocated inside the setIntervall call, but i dont know how to work around that. – daschl Apr 18 '11 at 13:23
  • What if you were to change it so that every N requests, it refreshes the browser page instead of making the AJAX call? – davidtbernal Apr 18 '11 at 15:32
  • Hi notJim! Actually we've tried that and at first it seemed to solve the problem but the customer told us that the problem manifests itself again (after a certain timeframe). I really wonder why the browsers don't clean up the memory for the old ones – daschl Apr 18 '11 at 17:59
  • Does the memory *grow without bounds* or achieve a stable value after a time? Does the data returned increase in size over time? Is Firebug or another tool which monitors net connections being used? –  Apr 21 '11 at 00:13
  • The test page I added is not truly equivalent, but if the problem is what you appear to say it is, then it shouldn't matter and both cases should reproduce it – Earlz Apr 21 '11 at 06:32
  • Also, what particular browsers are you concerned with? – Earlz Apr 21 '11 at 06:33
  • @pst: it grows slowly out of bounds, until the memory for the browser has reached a critical level where the user has to restart it, so the computer won't get totally unusable. I've profiled it with Firebug and Chrome Devtools. – daschl Apr 21 '11 at 13:32
  • @Earlz: you're right its not 100% the same but i trimmed the problem down to the call. I need to make it running at (at least) IE8 - that's what is used internally. (but i think the solution would fit all maybe) – daschl Apr 21 '11 at 13:33
  • as a note, i noticed that moving over to JSON from HTML reduces the ajax response size and therefore decreases the memory "jumps", but then i have to use something like mustache or handlebars to render a template with the json data (which is not a good choice for the short-term actually as it requires me to rewrite a good part of the app) – daschl Apr 21 '11 at 13:36
  • @moidaschl: Have you found a solution for this already? – Grodriguez Sep 10 '11 at 08:28

7 Answers7

5

Just a thought, instead of setInterval you should use setTimeout and then when it times out you set the timeout again.

That way you don't run the risk of the setInterval running off if you lose track of it for some reason:

I think also that the jquery ajax has a success callback which you can use as the point that you setTimeout. That way, as has been mentioned in this thread elsewhere, you dont end up overlapping requests.

(just checked and there is a success callback)

rtpHarry
  • 13,019
  • 4
  • 43
  • 64
  • hi rtpHarry, thanks for your feedback. As said in my comments above i tried this and it didnt change the memory load - sadly. Thanks anyway! – daschl Apr 21 '11 at 13:41
4

One possible problem with the same posted is that if the XHR requests take longer than the poll period (on average) there will be an increasing queue of pending requests. If the web-server itself starts to backlog requests this can turn into a vicious cycle.

To avoid this possible case, use a CPS-style coding where the next action is done using the appropriate callbacks. That is, don't start the next request until required (previous request complete: success or failure) -- this approach can still be used to create a manual request queue with a controllable size if numerous outstanding requests are required.

Also, make sure that unused objects are be eligible for reclamation as this is the "standard" cause of a "memory leak" in a GC language.

Happy coding.

The code in the post contains nothing that will inherently leak memory. It could possibly be an issue internally with jQuery, but this is just speculation. Additionally, tools like Firebug that monitor XHR/web requests can consume significant amounts of memory so it's something to check and make sure the behavior is not a Heisenberg.

Also, remember that increasing memory usage doesn't not indicate a memory leak unless it grows unbounded. A garbage collection cycle will only occur when the hosts feels like it.

  • Hi pst, thanks for your feedback and thoughts on this. I tried to use setTimeouts "on success" instead of setIntervals, but unfortunately this doesn't change the problem. I now think that it isnt really a memory leak, but it could be that when you load a bunch of HTML maybe it is attached or in some way bound to the DOM and never released. So (as said in my other comment above), maybe it's a good idea to load some JSON instead and render it. I also think that the problem wouldn't come out in production IF the users wouldn't have the app open 24/7. The production interval is 30 seconds.... – daschl Apr 21 '11 at 13:39
  • ...which should be enough time to release the old dom elements. Sadly it happens with IE too (where we don't have firebug actually). I wonder how the "Big players" like facebook or twitter handle this case in their environments (chat, realtime stream and so on). – daschl Apr 21 '11 at 13:40
0

you're polling sever after every second. Definitely it'll consume all of the memory. What however you can do is to set a predefined interval. Then after polling if the data is available fetch that and keep the interval same, but if after polling there is no data available yet, increment the interval to make sure next poll is delayed. This way you can reduce the load.

Naveed Ahmad
  • 3,176
  • 1
  • 15
  • 18
  • Hi Ahmad, thanks for your feedback. I said in my initial posting that this is set to 30 seconds in the production and just set to one second so that thep problem manifests faster. Also, I'm not able to add a delay because after 2 hours the possibility of new data is the same as of now. Thanks anyway! – daschl Apr 18 '11 at 12:32
  • @moidaschl Okay, apart from cleaning the memory (Garbage Collection) you must put indicators on each data load and should not load everything but only from that indicator. I hope you understand what I meant. – Naveed Ahmad Apr 18 '11 at 12:35
  • Hm not really, it would be great if you could explain that indicator stuff a bit more (or do you have some links available on that topic?) – daschl Apr 18 '11 at 12:43
  • You can set a variable with the latest row ID and the next ajax fetch should continue where row ID is greater than the current row ID. – Naveed Ahmad Apr 18 '11 at 12:51
  • Ok if i understand you correctly this would result in more, but smaller ajax queries, right? The problem i face here is that i have n entries to load, but there could also be n+1 or n-1 entries on the next fetch, so i need to pull the whole dataset on every request cycle. – daschl Apr 18 '11 at 12:54
  • Assuming that the fetched entries will remain same, and won't get updated externally, you can keep track of their IDs and send that back to the server to test against the matched records and don't send existing entries. Just an idea. – Naveed Ahmad Apr 18 '11 at 12:58
  • This sounds great, but I can't check if they are the same from the client side. What I do to reduce the data loaded over the wire is to send a hash with it and the client on the next requests sends this hash back. wenn the server sees that the data hasn't changed, he simply returns nothing. This way i could reduce the leak a bit but it doesn't solve it. – daschl Apr 18 '11 at 13:03
  • Exactly, you can avoid loading the whole data again and again but that won't solve your actual problem. May be you're trying to create to many DOM elements at runtime on each data fetch. Try avoiding that and also loading too many images or at least use compressed images. This can help. – Naveed Ahmad Apr 18 '11 at 13:09
  • Hi, thanks again :). Actually I'm not loading any images. In my production app i load a HTML data (~100KB on each fetch), then remove the old entries and add the new ones to the DOM. The strange thing is that in this example above NO data mangling happens and the memory still cramps up... – daschl Apr 18 '11 at 13:11
0

Have a look at this post - might help ?

How can I force Javascript garbage collection in IE? IE is acting very slow after AJAX calls & DOM manipulation

(Suggests using specific JQuery methods that apparently try to avoid memory leaks: and using a 'delete' command to try and free up space).

Community
  • 1
  • 1
monojohnny
  • 5,894
  • 16
  • 59
  • 83
  • Thanks for the link. Actually it happens in all browsers, not just IE. At first i thought my data manipulation is producing this memory increase, but then i just ran the AJAX query (as you can see in the example above) and don't do anything else and it still goes up. I also tried the html() and delete stuff but it doesn't help in this case. Thanks anyway! – daschl Apr 18 '11 at 12:38
  • Note, using delete is a bad idea, if you're reading this answer in or after 2013, don't use it – Benjamin Gruenbaum Mar 18 '13 at 11:42
0

You will increase memory usage since you are loading more data on every request.

Clicky:

How to free memory after an Ajax request

Memory leak involving jQuery Ajax requests

Community
  • 1
  • 1
Calum
  • 5,308
  • 1
  • 22
  • 27
  • Hi Calum, thanks for the links. The first one just says I can't do nothing about it, which is not an option here :D. The second one involves manipulating the DOM which i don't do here and the memory still increases. IN my production app i manipulate them DOM, and maybe this $.fn.removefromdom helps to reduce it a bit. thanks :) – daschl Apr 18 '11 at 12:50
0

According to this jQuery forum thread:

...
success: function(data) {

    // the line below prevents the leak, apparently
    document.getElementById("theContainer").innerHTML = ""; 
    $("#theContainer").html(data);
}
karim79
  • 339,989
  • 67
  • 413
  • 406
  • hm i tried this, and also changed the time to 10 seconds but you can see clearly in the task manager that on every request the mem goes up but in the time interval the memory does not get removed. – daschl Apr 18 '11 at 13:38
  • I does not change anything. – Marek Bar Sep 23 '15 at 13:11
0

Have you tried reusing the variables inside your calls? Meaning instead of making new variables where you handle the response, just use the same ones. Say 

window.myvar = response["myvar"];

It it's really a leak, this might help the garbage collector. It would be nice to see the actual code of what you're doing though.

McTrafik
  • 2,843
  • 3
  • 29
  • 31