Waiting for an async HTTP request to complete

Question

This question has been asked in various ways already here, and I understand the problem pretty well. The general solution is to invoke callbacks for handling HTTP results, but I'm specifically wanting to avoid that.

I imagined I could use a "are we done yet" flag to keep things working, but this isn't working as expected.

//
//  Test the IP the comment came-from.
//
exports.testJSON = function ( obj )
{
    var ip     = obj['ip'] || "";
    var result = null;
    var done   = false;

    //
    //  The URL request we're going to make
    //
    var options = {
        host: 'www.stopforumspam.com',
        port: 80,
        path: '/api?ip=' + ip
    };

    //
    // A GET request
    //
    var re = http.request(options, function(res) {
        var str = '';

        res.on('data', function(chunk) {
            console.log( "Got data " + chunk );
            str += chunk;
        });
        res.on('end', function() {
            console.log( "Got END" );
            result = "..... ";
            done = true;
        });
      }).on('error', function(e) {
         done = true;
         console.log("Got error: ", e);
     });

   re.end();

   while( ! done ) {
   }

   return( result );
};

Sadly this doesn't work - the busy look just spins indefinitely, and I see no console-logging to indicate that I'm receiving data.

Adding in "process.nextTick()" to the "while(!done){}" loop makes no difference either.

Surely I don't need to rework my whole plugin system to cope with a different approach, and the callback updating the "done" flag will work, somehow?

I'm sorry to say, but if you're trying to use a synchronous loop to check for an asynchronous result you don't understand the problem pretty well. What it seems you're trying to do, _return_ a result from an asynchronous call, is not possible in javascript (afaik). Any synchronous code will either block the entire thread (preventing the request to be made) or return before the request is made. — Andreas Hultgren, Sep 10 '13 at 14:55
I suppose one could make it work by hacking some [process.binding](http://stackoverflow.com/questions/17336779/if-nodejs-uses-non-blocking-io-how-is-fs-readfilesync-implemented), but I wouldn't recommend it. — Andreas Hultgren, Sep 10 '13 at 14:58
Yes I'm porting a system from perl + XML::RPC to HTTP-JSON consuming node.js server. Andreas - yes, you're right. What I meant was I understand why things are the way they are, and because I'm using a naive plugin architecture I cannot easily adapt to using a callback to return the result. The specific problem is the function end is reached before the callback completes - which is obvious, but I had hoped to work around that and I see that I cannot :( — , Sep 10 '13 at 15:00

score 2 · Accepted Answer · answered Sep 10 '13 at 15:03

If your whole plugin system is using JavaScript, reworking it probably wouldn't be a bad idea. If you don't want to deal with asynchronous code using callbacks, you can look into promises as another solution for managing asynchronous code. The library "q" may be a good place to start with that.

An alternative would be to use node-fibers. I've never used it before, but the examples are straightforward. Take a normal callback pattern for example:

console.log('waiting 2 seconds...');
setTimeout(function() {
    console.log('Finished!');
}, 2000);

A similar example using node-fibers:

var Fiber = require('fibers');

function wait(ms) {
    var fiber = Fiber.current;
    setTimeout(function() {
        fiber.run();
    }, ms);
    Fiber.yield();
}

Fiber(function() {
    console.log('waiting 3 seconds...');
    wait(1000);
    console.log('waiting 2 seconds...');
    wait(1000);
    console.log('waiting 1 seconds...');
    wait(1000);
    console.log('done!');
}).run();

There is a little wrapping to be done to get node-fibers to function, but now you can avoid dealing with callbacks and use async code in a synchronous manner.

It's worth mentioning why his solution doesn't work - his node.js code runs in a single thread, and the while loop holds control of the thread forever, so the callback never has a chance to be called. — Aaron Dufour, Sep 10 '13 at 15:05
Indeed. It looks like I have to look at some of the mpm promise/queue modules and see if I can find a good fit. — , Sep 10 '13 at 15:08
I have used the [async](https://github.com/caolan/async) library in the past - may be worth taking a look at as well. — Jay, Sep 10 '13 at 15:10

Waiting for an async HTTP request to complete

1 Answers1