NodeJS async/await + recursion on express request

Question

I have a function called GetAllData() that calls GetPurchaseData, which recursively calls itself until it loads all the data. In

async function GetAllData(){
    console.log("starting loading purchase data");
        await GetPurchaseData();
        console.log("purchase data loaded")        
}

async function GetPurchaseData(){
         return new Promise(async function (resolve,reject){
            var Headers = {
                ....
            }
            await request({url: xxx, headers: Headers },async function(error, response, body) {
                    var tmp = JSON.parse(body)      
                    _.forEach(tmp.Purchases, p => purchaseData.push(p));                          

                    if (response.headers.pagination){
                        return await GetPurchasePaginatedData()
                    }
                    else{
                        console.log("done loading....")
                        return resolve("done")
                    }
            });
        })
}

Node JS prints the following output:

starting loading purchase data 
done loading....

but it never comes back up to GetAllData to print

purchase data loaded

it almost seems like its stuck in function, but my opinion is that somehow the line of "return resolve("done")" does not get back up to the initial call to actually mark the Promise as complete.

One thing to note is that you shouldn't need to explicitly return a `Promise` from an `async function`, `async` dictates that your function will return a `Promise` no matter what. — zero298, Aug 31 '17 at 18:21
Also, does `request` take a `Promise` or a callback? Last time I used it it wanted callbacks in standard node.js fashion. If that's the case you'll need to wrap `request` or find a library that works in `Promises`. — zero298, Aug 31 '17 at 18:23
i am wrapping request by doing return new Promise( ... request() ... ) am i not? — scorpion5211, Aug 31 '17 at 18:26
Which **exact** request library are you using? If it is the regular one, it only works in callbacks which you aren't wrapping correctly. Since you are working with `Promises` and `async/await` I would actually recommend using one of the wrapper libraries that `request` provides. You can choose whichever from [`request`'s GitHub](https://github.com/request/request#promises--asyncawait). — zero298, Aug 31 '17 at 18:35
@zero298 ha!! interesting! i am using a regular request library. can you post an example of correct way of wrapping? — scorpion5211, Aug 31 '17 at 18:43

score 1 · Answer 1 · answered Aug 31 '17 at 19:32

Avoid the async/await Promise constructor antipattern (see also here), and avoid passing an async function as a regular callback - you need to use the Promise constructor to promisify an existing callback API!

async function GetPurchaseData() {
    var headers = {…};
    var promise = new Promise((resolve,reject) => { // not async!
        request({url: xxx, headers}, (error, response, body) => { // not async!
            if (error) reject(error);
            else resolve({response, body});
        });
    }); // that's it!

    var {response, body} = await promise;
    for (var p of JSON.parse(body).Purchases)
        purchaseData.push(p));                          

    if (response.headers.pagination) {
        return GetPurchasePaginatedData()
    } else {
        console.log("done loading....")
        return "done";
    }
}

score 0 · Answer 2 · answered Aug 31 '17 at 18:13

I don't have much experience with async/await, but from what I've read, shouldn't the code look something like this.

async function GetAllData(){
    console.log("starting loading purchase data");
        await GetPurchaseData();
        console.log("purchase data loaded")        
}

async function GetPurchaseData(){
        let body = await request({url: xxx, headers: Headers })

        var tmp = JSON.parse(body)      
        _.forEach(tmp.Purchases, p => purchaseData.push(p));                          

        if (response.headers.pagination){
            return await GetPurchasePaginatedData()
        }
        else{
            console.log("done loading....")
            return "done"
        }
}

that was my initial approach except instead of `let body` i was doing `let response` then i was splitting up response into `response.headers` and `response.body`, but it didnt work. And by "didnt work" i mean it didnt actually await for the response to come back — scorpion5211, Aug 31 '17 at 18:29

NodeJS async/await + recursion on express request

2 Answers2