Using promises/callbacks to wait for function to finish in javascript

Question

I have a function which creates a database object out of three arrays. The arrays are filled in an each loop, one of the arrays relies on the value in the same iteration of the loop.

The dependent array uses the requests library and the cheerio library to grab a string to populate the array with.

Currently the dependent array fills with nulls which I think is because the loop is not waiting for the request to be returned.

I am still learning and would like to get this to work without direct blocking to keep things asynchronous so I'm looking into promises/callbacks.

This is being done server-side but from what I've seen in cheerios docs there is no promises capability.

Here's what I have so far. (getFile() is the function that isn't filling the 'c' array, it also depends on the current value being put into 'b'). I do know that the getFile function gets the correct value with a console log test, so the issue must be in the implementation of filling 'c'.

addToDB() is a function which saves a value into mongoDB, from testing I know that the objects are correctly being put into the db, just the c array is not correct.

function getInfo(path) {
  $(path).each(function(i,e) {
    a.push(...)
    b.push(value)
    c.push(getFile(value))
  })
  var entry = new DB...//(a,b,c)
  addToDB(entry);
}

function getFile(...) {
  request(fullUrl, function (err, resp, page) {
    if (!err && resp.statusCode == 200) {
      var $ = cheerio.load(page); // load the page
      srcEp = $(this).attr("src");
      return srcEp;
    } // end error and status code
  }); // end request
}

I've been reading about promises/callbacks and then() but I've yet to find anything which works.

Answer 1

First, you have to get your mind around the fact that any process that relies, at least in part, on an asynchronous sub-process, is itself inherently asynchronous.

At the lowest level of this question's code, request() is asynchronous, therefore its caller, getFile() is asynchronous, and its caller, getInfo() is also asynchronous.

Promises are an abstraction of the outcome of asynchronous processes and help enormously in coding the actions to be taken when such processes complete - successfully or under failure.

Typically, low-level asynchronous functions should return a promise to be acted on by their callers, which will, in turn, return a promise to their callers, and so on up the call stack. Inside each function, returned promise(s) may be acted on using promise methods, chiefly .then(), and may be aggregated using Promise.all() for example (syntax varies).

In this question, there is no evidence that request() currently returns a promise. You have three options :

• discover that request() does, in fact, return a promise.
• rewrite request() to return a promise.
• write an adapter function (a "promisifier") that calls request(), and generates/returns the promise, which is later fulfilled or rejected depending on the outcome of request().

The first or second options would be ideal but the safe assumption for me (Roamer) is to assume that an adapter is required. Fortunately, I know enough from the question to be able to write one. Cheerio appears not to include jQuery's promise implementation, so a dedicated promise lib will be required.

Here is an adapter function, using syntax that will work with the Bluebird lib or native js promises:

//Promisifier for the low level function request()
function requestAsync(url) {
    return new Promise(function(resolve, reject) {
        request(url, function(err, resp, page) {
            if (err) {
                reject(err);
            } else {
                if (resp.statusCode !== 200) {
                    reject(new Error('request error: ' + resp.statusCode));
                }
            } else {
                resolve(page);
            }
        });
    });
}

Now getFile(...) and getInfo() can be written to make use of the promises returned from the lowest level's adapter.

//intermediate level function
function getFile(DOMelement) {
    var fullUrl = ...;//something derived from DOMelement. Presumably either .val() or .text()
    return requestAsync(fullUrl).then(function (page) {
        var $ = cheerio.load(page);
        srcEp = $(???).attr('src');//Not too sure what the selector should be. `$(this)` definitely won't work.
        return srcEp;
    });
}

//high level function
function getInfo(path) {
    var a = [], b = [], c = [];//presumably

    // Now map the $(path) to an array of promises by calling getFile() inside a .map() callback.
    // By chaining .then() a/b/c are populated when the async data arrives. 
    var promises = $(path).map(function(i, e) {
        return getFile(e).then(function(srcEp) {
            a[i] = ...;
            b[i] = e;
            c[i] = srcEp;
        });
    });
    //return an aggregated promise to getInfo's caller,
    //in case it needs to take any action on settlement.
    return Promise.all(promises).then(function() {
        //What to do when all promises are fulfilled
        var entry = new DB...//(a,b,c)
        addToDB(entry);
    }, function(error) {
        //What to do if any of the promises fails
        console.log(error);
        //... you may want to do more.
    });
}