Node.JS - How to identify which async HTTP GET response is returning data in a callback?

Question

I'm trying to follow the NodeSchool tutorials on async principles. There is a lesson with the requirement to:

make asynchronous GET requests to 3 URLs
collect the data returned in HTTP responses using callbacks
print the data collected from each response, preserving the order of the correlating input URLs

One way I thought of doing this was to make 3 separate callbacks, each writing to its own dedicated buffer. But this seemed like a poor solution to me, because it requires code duplication and is not scalable. However, I am stuck on figuring out another way to get a callback to remember "where it came from" for lack of a better term. Or what order it was called in, etc.

I feel like I'm close, but I've been stuck at 'close' for long enough to look for help. Here's the closest I've felt so far:

var http = require('http');

var replies = 0;
var results = ['','',''];
var running = '';

for(var i = 2; i < 5; i++) {
    http.get(process.argv[i], function (response) {
        response.setEncoding('utf8');
        response.on('data', handleGet);
        response.on('error', handleError);
        response.on('end', handleEnd);
    });
}

function handleGet(data) {
    running += data;
}

function handleError(error) {
    console.log(error);
}

function handleEnd() {
    results[replies] = running;
    running = '';
    replies++;
    if(replies === 3) {
        for(var i = 0; i < 3; i++) {
            console.log(totals[i]);
        }
    }
}

How can I get the callback to recognize which GET its response is a response to?

Note: the assignment specifically prohibits the use of 3rd party libs such as async or after.

Edit: another thing I tried (that also obviously failed), was inlining the callback definition for handleGet like so, to try and preserve the 'index' of the callback:

response.on('data', function(data) {
    results[i-2] += data;
});

On execution, this always indexes to results[3] because the async callbacks don't happen until after the for loop is already long done. (Actually I'm not sure why the value of i is preserved at 5 at all, since the completed for loop would go out of scope, as I understand it... I would have thought it'd be undefined in retrospect.)

see possible duplicate [JavaScript closure inside loops – simple practical example](http://stackoverflow.com/q/750486/1048572). Use separate callbacks without code duplication. — Bergi, Feb 15 '15 at 18:44
Thanks, this is the behavior I've been trying to duplicate, but the problem is that I can't get the value of the for(...) iterator into the nested callback. The first callback I can understand how this func[] generator works, but I don't see how to get the data down the chain to further callbacks. — Steverino, Feb 15 '15 at 19:01
It's just the same for these - each request gets its own callbacks. — Bergi, Feb 15 '15 at 19:04
Thanks for directing me, this is a big turning point in my understanding of JS and NodeJS. — Steverino, Feb 15 '15 at 19:19

skay- · Accepted Answer · 2015-02-15T19:38:00.603

1

I will suggest using your second solution but passing the value of i and creating another function which encloses that variable. By this I mean the following:

(function(){
  var index = i;
  http.get(process.argv[i], function (response) {
        response.setEncoding('utf8');
        response.on('data', handleGetFrom(index));
        response.on('error', handleError);
        response.on('end', handleEnd);
    })
}());

and the handleGetFrom:

var handleGetFrom = function(i) {
    return function(data) {
     results[i-2] += data;
    }
}

Edited my original answer.

edited Feb 15 '15 at 19:38

answered Feb 15 '15 at 19:14

skay-

1,546
3
16
26

This is similar to my second attempt. It does not seem to work, as we have established that the function handleGetFrom(i) would not get called during the for loop, while 'i' has the value we expect. Bergi pointed me toward the right answer in his comments above. – Steverino Feb 15 '15 at 19:31
oh i see, i missed the outer function... i will edit my answer – skay- Feb 15 '15 at 19:33
This solution does work, although I'm trying to be clear on why. Is it because the value of variable 'index' is preserved by the JS function scope of the anonymous function you are wrapping everything in? (Whereas the for loop does not have scope in JS?) – Steverino Feb 15 '15 at 20:07

score 0 · Answer 2 · answered Feb 15 '15 at 18:43

0

You can use the 'this' object:

var http = require('http');

http.get('http://www.google.com', function (response) {
    response.setEncoding('utf8');
    response.on('data', function(chunk) {
        console.log(this); // Holds the respons object
        console.log(this.headers.location); // Holds the request url
    });
});

answered Feb 15 '15 at 18:43

Ben Diamant

6,186
4
35
50

This is returning 'undefined' for me. – Steverino Feb 15 '15 at 18:51
It's pages of data, but here is one response: (sorry editing) { date: 'Sun, 15 Feb 2015 18:56:29 GMT', expires: '-1', 'cache-control': 'private, max-age=0', 'content-type': 'text/html; charset=ISO-8859-1', 'set-cookie': [ 'PREF=ID=f6d95e9ae8ebfdce:FF=0:TM=1424026589:LM=1424026589:S=f5gC88eIdqdTGTF1; expires=Tue, 14-Feb-2017 18:56:29 GM T; path=/; domain=.google.com', 'NID=67=p15PVifZ9eZ0I3eln2jFGAc2j54 ... (too long to fit in comment) Actually, looking at the object, none of the properties is labeled location. Some (not all) of the returned objects contain a location property. – Steverino Feb 15 '15 at 18:58

Node.JS - How to identify which async HTTP GET response is returning data in a callback?

2 Answers2