7

I'm trying to make a web crawler to fetch products from certain sites to reduce my memory usage (I've got a memory leak somewhere i haven't found). So I'm trying to send arguments to a callback asynchronously in order to terminate the current context.

This is where I'm at:

var big = html.get(url);
callback(big);
big = null; // this isn't going to get erased until after big is done is it?

This is what I've tried:

var big = html.get(url);
process.nextTick(function() {callback(big)}); // this seems wrong to me.
big = null;
limoragni
  • 2,716
  • 2
  • 32
  • 50
leech
  • 8,293
  • 7
  • 62
  • 78
  • You can't "erase" big until you've finished using it. If you don't want it in memory, you could write it to disk for processing later but if its only one CPU cycle later then that's kinda pointless (disk IO is pretty slow). You may need to give an idea of what you're doing in "callback" for an appropriate answer. – Shakakai Aug 23 '11 at 22:30
  • it's a crawler that takes pages and looks for certain information, like products on a shop page. So it recursively follows links and fetches useful info. – leech Aug 23 '11 at 22:56

3 Answers3

13

process.nextTick() doesn't support passing arguments like setTimeout() does. You will just need to make a function that passes the arguments.

So instead of:

setTimeout(myfunc, 0, 'arg1', 'arg2')

Use this:

process.nextTick(function() { myfunc('arg1', 'arg2') })

You can also use a function to build your function.

Finally, setTimeout doesn't support specifying the object that the function gets called on, so it's limited in its usefulness.

Benjamin Atkin
  • 14,071
  • 7
  • 61
  • 60
  • 2
    Wow, that is one little-documented feature! BTW, on my browser and my installation of Node, it's `setTimeout(myfunc, 0, arg1, arg2)` (i.e., no list and no quotation marks. Either way, better to use closures for both nextTick and setTimeout. – Michael Lorton Aug 24 '11 at 03:56
  • D'oh. Thanks for pointing that out. I got confused by the square brackets indicating they were optional. I thought they indicated it was an array. https://developer.mozilla.org/en/window.setTimeout – Benjamin Atkin Aug 24 '11 at 04:19
12
var big = html.get(url);
process.nextTick(function() {callback(big); big = null; }); 

If this style isn't obvious to you, then you should study closures more.

(incidentally, setting big to null is probably unnecessary here; once the callback finishes, big is going out of scope.)

Edit: In honor of the six-year anniversary of this question, here is an ECMA-6 version of the same thing:

const big = html.get(url);
process.nextTick(() => callback(big)); 
Michael Lorton
  • 43,060
  • 26
  • 103
  • 144
2

Although process.nextTick() doesn't directly support passing arguments, there is a shortcut that achieves the same thing.

var big = html.get(url);
process.nextTick(callback.bind(null, big));

Note that this isn't functionally any different from the explicit closure answers already given (minus manually setting big = null; instead of letting it go out of scope on its own), but I feel it makes the code a little easier to read.

bind() is actually a much more powerful tool than is shown here; read more (including some examples) at the MDN documentation.

Details: The first parameter to bind() is the value which will be assigned to this during the execution of callback(), which is assigned to null here since it looks like it wouldn't be used in this case. Any additional parameters (only big in this case) will be passed as arguments to callback when it is called later, via a technique called "partial function application".

zacronos
  • 1,487
  • 1
  • 12
  • 13