making http requests in nodejs loop

Question

Finding it almost impossible to capture the response of http requests in a loop as an array. I can see the array in console.log but when I pass the array to be the response of http server I get a blank array. What am I doing wrong , or are there better ways to do this?

Code :

router.route('/uprns').post(function(request, response){
  response.setHeader('content-type', 'application/text');

  console.log('first element from the array is '+request.body.UPRNS[0]);
  console.log('Number of items in array is '+request.body.UPRNS.length);

if (request.body.UPRNS.length == 0) {
         response.send( 'no UPRNS in request' );
    }

  var output = [];
  var obj = '';

  for( var i = 0; i < request.body.UPRNS.length; i++) {

    obj = request.body.UPRNS[i];

    //Make  HTTP calls to     
    var options = {
      host: 'orbisdigital.azure-api.net',
      path: '/nosecurity/addresses?uprn='+obj // full URL as path
    };

    callback = function(res) {    
      res.on('data', function (chunk) {
        output.push(chunk.toString());
      });

      //the whole response has been recieved
      res.on('end', function () {
        console.log(output);
      });
    }

    Https.request(options, callback).end();
  }

  response.send(output);

});

I know there is a lot of talk about blocking process in a for loop , but there is no definitive recommended way to deal with http calls in a loop. Thank you .

The order of elements in the output array is important? Should he comply with the order of http-requests? — stdob--, Apr 29 '16 at 08:32
The order is not important in my case , but would like to know how to achieve the order as well . — noexpert, Apr 29 '16 at 09:18

score 0 · Accepted Answer · edited May 23 '17 at 11:59

Here is the code. See the code for the added comments. Do some reading on asynchronous programming with node.js, here's a starter.

router.route( '/uprns' ).post( function ( request, response ) {
    response.setHeader( 'content-type', 'application/text' );
    console.log( 'first element from the array is ' + request.body.UPRNS[ 0 ] ); // your  1st element in  JSON array.

    console.log( 'Number of items in array is ' + request.body.UPRNS.length );
    var output = [];
    var obj = '';

    for ( var i = 0; i < request.body.UPRNS.length; i++ ) {

        obj = request.body.UPRNS[ i ];

        console.log( obj );

        //Make  HTTP calls to

        var options = {
            host: 'orbisdigital.azure-api.net',
            path: '/nosecurity/addresses?uprn=' + obj // full URL as path
        };

        Https.request( options, callback ).end();

    }

    var countResponses = 0;
    // Don't make functions in a loop, so I moved this function down
    // here.
    function callback( res ) {

        res.on( 'data', function ( chunk ) {
            output.push( chunk.toString() );
        });

        // Handles an error
        request.on('error', function(err) {
          console.error(err.stack);
          response.statusCode = 500; // or what ever.
          response.send(500, 'there was an error');
        });

        //the whole response has been recieved
        res.on( 'end', function () {
            console.log( output );
            countResponses++;
            if (countResponses === request.body.UPRNS.length) {

                // Previously this code was executed directly 
                // after the loop finished.  It did not wait for
                // all the responses, so it sent the empty response.
                // However, the other console.log(output) statements
                // were called after this.
                //
                // There is a bug here that if request.body.UPRNS.length
                // is zero, then the user will never get a response.  I 
                // let you fix this up :).
                response.send( output );
            }
        } );

    }

} );

@stdob: Yes, good point, I will add. Also when request.body.UPRNS.length is zero, Ieave this up the person asking. — psiphi75, Apr 29 '16 at 08:49
Great , worked like a charm . will work on handling null requests.Thanks for your help. — noexpert, Apr 29 '16 at 09:19
added a simple if condition to deal with empty input array . added to original code. if (request.body.UPRNS.length == 0) { response.send( 'no UPRNS in request' ); } — noexpert, Apr 29 '16 at 11:49

score 0 · Answer 2 · answered Apr 29 '16 at 08:52

0

Better way to handle such scenario is to use async.js instead of for loops. https://github.com/caolan/async

answered Apr 29 '16 at 08:52

Phagun Baya

2,127
1
18
27

making http requests in nodejs loop

2 Answers2