making multiple api endpoint calls from array nodejs

Question

I'm having trouble making calls to API endpoints from an array. The array contains values that I need to feed into the endpoint in order to get data that I need. The trouble I am having is when I console log the data from the api call I receive JSON text. However, when I try to parse the text and get a specific value from the JSON text it is not giving me the results. Here is the code I am working on:

var http = require("http");
var areaCodes = ["'0071950')", "'0072850')", "'0073450')", "'0075700')", "'0076450')", "'0078700')", "'0900001')", "'0900000')"];
var results = [];
var encode = [];

    //source: http://stackoverflow.com/questions/17811827/get-a-json-via-http-request-in-nodej
for(var i = 0; i < areaCodes.length; i++){
    var realmStatus = "http://api.dol.gov/V1/Statistics/OES/OE_SERIES/?KEY=XXXXXXXXXXXXXXXXXXXXXXXXXX&$filter=(OCCUPATION_CODE eq '151131' ) and (AREA_CODE eq " + areaCodes[i];
     encode.push(encodeURI(realmStatus));

var options = {
        host: 'api.dol.gov',
        path: encode[i],
        type: 'GET',
        dataType: 'json',
        headers: {'accept' : 'application/json'}
};
console.log("Start");
var x = http.request(options,function(res){
    console.log("Connected");
     var str = '';
    res.on('data', function(chunk) {
        str += chunk;
    });
    res.on('data',function(data){
        if(res.statusCode == 200){
            //console.log("res status is good");
            try{
                 var dataA = JSON.parse(str);
                //run a for loop 
                for(var h = 0; h < dataA.d.results.length; h++){
                    var seriesNum = dataA.d.results[h].SERIES_ID; 
                    array.push(seriesNum);
                    console.log(seriesNum);
                }
                 //end for
            }catch(e){
                console.log('Error parsing JSON');
            }
        }
    });
});
x.end();
}//end for loop

I'd appreciate if someone would be willing to explain if there is anything I did incorrectly, or if there is something I am missing. Thanks!

score 0 · Answer 1 · answered Apr 18 '17 at 21:03

0

It looks like your second res.on('data') (the one with the try...catch) should actually be a res.on('end'), which is fired once you've gotten the whole file.

Check out the example in the official Node.js documentation: https://nodejs.org/api/http.html#http_http_get_options_callback. Notice that they use res.on('data') once and res.on('end') once.

Beyond that, off-hand it looks like you should at least get something parsed.

answered Apr 18 '17 at 21:03

samanime

25,408
15
90
139

hey thanks for the quick response, however, that does not work. It's odd because I used the same code for another endpoint and it works fine. The only difference is that in the other endpoint I did not use an array to store values to concatenate the endpoint. – learningToCode Apr 18 '17 at 21:52
The problem is that when I run the program I get "Error parsing JSON" meaning that the `try` block is not being executed. This is not a problem when I make a call for my other endpoint – learningToCode Apr 18 '17 at 22:02
I think the reason is because you're calling that second `'data'` immediately after the first one. Depending on the size of the payload, you may get the entire JSON in just one data (which is probably why other endpoints work), but for this one, that payload requires multiple `data` calls. However, you're calling this after the first, so you probably only have like half of the JSON, which is why it throws an error. – samanime Apr 20 '17 at 18:56
the funny thing is that it would work sometimes, for instance a couple hours after the initial call it didn't work but then it gave me the results i wanted.. weird – learningToCode Apr 20 '17 at 19:14
That would lend more to what I said. If the payload got small enough (or fast enough) to come in on one data call, it'd work. If it's larger, the `.on('data')` would have to be called multiple times, but your code would try to parse JSON after the first. – samanime Apr 20 '17 at 21:07
I'm pretty sure my answer should resolve the problem. Give it a try, then debug by adding some more console logs between each statement and see what you get and when. – samanime Apr 23 '17 at 08:48

making multiple api endpoint calls from array nodejs

1 Answers1