Node.JS: How to pass variables to asynchronous callbacks?

Question

I'm sure my problem is based on a lack of understanding of asynch programming in node.js but here goes.

For example: I have a list of links I want to crawl. When each asynch request returns I want to know which URL it is for. But, presumably because of race conditions, each request returns with the URL set to the last value in the list.

var links = ['http://google.com', 'http://yahoo.com'];
for (link in links) {
    var url = links[link];
    require('request')(url, function() {
        console.log(url);
    });
}

Expected output:

http://google.com
http://yahoo.com

Actual output:

http://yahoo.com
http://yahoo.com

So my question is either:

How do I pass url (by value) to the call back function? OR
What is the proper way of chaining the HTTP requests so they run sequentially? OR
Something else I'm missing?

PS: For 1. I don't want a solution which examines the callback's parameters but a general way of a callback knowing about variables 'from above'.

JohnnyHK · Accepted Answer · 2012-11-04T19:43:31.017

50

Your url variable is not scoped to the for loop as JavaScript only supports global and function scoping. So you need to create a function scope for your request call to capture the url value in each iteration of the loop by using an immediate function:

var links = ['http://google.com', 'http://yahoo.com'];
for (link in links) {
    (function(url) {
        require('request')(url, function() {
            console.log(url);
        });
    })(links[link]);
}

BTW, embedding a require in the middle of loop isn't good practice. It should probably be re-written as:

var request = require('request');
var links = ['http://google.com', 'http://yahoo.com'];
for (link in links) {
    (function(url) {
        request(url, function() {
            console.log(url);
        });
    })(links[link]);
}

edited Nov 04 '12 at 19:43

answered Nov 04 '12 at 19:11

JohnnyHK

305,182
66
621
471

3

It's not about scope. It's about closures. Lots of other languages don't have block scope yet don't face this issue due to lack of closures. – slebetman Nov 04 '12 at 20:02
3

The point is that if JavaScript did support block scope, then the closure access to `url` in the callback function of the OP's code would work because each iteration of the loop would get its own `url` variable (like in C#). – JohnnyHK Nov 04 '12 at 20:17
1

The require is just in there for brevity, in my code all requires are at the beginning. – Marc Nov 05 '12 at 09:14
If I don't use forloop within the callback it will return url undefined. – Aero Wang Mar 12 '18 at 02:27

score 13 · Answer 2 · answered May 01 '16 at 20:50

13

Check this blog out. A variable can be passed by using .bind() method. In your case it would be like this:

var links = ['http://google.com', 'http://yahoo.com'];
for (link in links) {
var url = links[link];

require('request')(url, function() {

    console.log(this.urlAsy);

}.bind({urlAsy:url}));
}

answered May 01 '16 at 20:50

Tjs

843
10
17

3

I like this method more as it's much less verbose than wrapping something in a function. – Levi Roberts Jul 21 '16 at 18:15
1

This one works without forloop! – Aero Wang Mar 12 '18 at 02:38

score 7 · Answer 3 · edited May 23 '17 at 11:46

7

See https://stackoverflow.com/a/11747331/243639 for a general discussion of this issue.

I'd suggest something like

var links = ['http://google.com', 'http://yahoo.com'];

function createCallback(_url) {
    return function() {
        console.log(_url);
    }
};

for (link in links) {
    var url = links[link];
    require('request')(url, createCallback(url));
}

edited May 23 '17 at 11:46

Community

1
1

answered Nov 04 '12 at 19:15

BobS

2,588
1
15
15

There are lots of better links to give on StackOverflow to explain the issue. This one for example: http://stackoverflow.com/questions/3572480/please-explain-the-use-of-javascript-closures-in-loops/3572616#3572616 – slebetman Nov 04 '12 at 19:59

Node.JS: How to pass variables to asynchronous callbacks?

3 Answers3

Linked