async.mapLimit with Promise

Question

I'm trying to use the async module (v3) and especially the async.mapLimit method to submit a limited number of parallel asnychronous requests. This works well with callbacks in the following (simplified) sample code:

async = require('async');
async.mapLimit(['1','2','3','4','5'], 3, function(num, callback){
setTimeout(function(){
    num = num * 2,
    console.log(num);
    callback(null, num);
}, 
4000);  
},function(err, results){
    console.log(results);
});

As result I get the single values and finally the array with all values in 'results':

[2,4,6,8,10]

Now what I'm struggling with is to use the Promise based version of this method. The documentation says it returns a Promise if I don't supply a callback. How can I change this callback-based code into using Promises?

I tried e.g. this, but it only shows the first set of requests (this is updated after the first suggestions in the comments):

let numPromise = async.mapLimit(['1','2','3','4','5'], 3, function(num, callback){
    setTimeout(function(){
        num = num * 2,
        console.log(num);
        callback(null, num);
    }, 
    4000); 
});
Promise.all([numPromise]) //always calls .then immediately, without result
.then((result) => console.log("success:" + result))
.catch(() => console.log("no success"));

I do get back all single values correctly, but the '.then' executes immediately and gives me 'Success' with an empty 'result' variable.

Note: I have seen this thread ES6 Promise replacement of async.eachLimit / async.mapLimit of course, but I don't think that answers my question.

PS: I have found other solutions to this problem already, but I am really interested in how to use this module properly (with short, clean code).

You say, "it returns a Promise if I don't supply a callback" but you are still supplying a callback. Leave it out and chain `.then()` instead. Then's callback will be passed `num`. Inside the callback, promisify the setTimeout and return the resulting promise. I'd write an answer but I'm on the wrong device. — Roamer-1888, Aug 24 '19 at 04:02
Thanks a lot for your thought on this. However, if I'm not mistaken, then I'm not supplying the callback parameter, but the 'Iteratee' (which is also an async function). This should be correct, but I'll give it a try. — Erik Reder, Aug 24 '19 at 07:41
You are right, third argument is iteratee. Fourth argument, the main callback, is the one that if left out causes `async.mapLimit()` to return promise. The iteratee still needs its callback, which needs to be called as in your first code block. — Roamer-1888, Aug 25 '19 at 02:01
You don't need `Promise.all()`, just `numPromise.then(function(result) {...})`: — Roamer-1888, Aug 25 '19 at 02:05
@Roamer-1888 thx a lot, re-adding the callback that did the trick! However, just using `numPromise.then...` gives me an 'undefined' error. I don't really understand this, though, but am glad that `Promise.all()` works at least. If you add this as answer, I'll of course accept it. — Erik Reder, Aug 26 '19 at 07:03
Actually, there's one issue I forgot to mention. In the callback example, the final 'result' gives me the array of all numbers, i.e. [2,4,6,8,10]. In the Promise version, it immediately says 'success', but with an empty array of course. So, how I can access the result set correctly here? Will update my initial post. — Erik Reder, Aug 26 '19 at 09:47
That's a good question and the documentation doesn't help because it is utter rubbish. You have a clue in that `numPromise.then()` gives you an 'undefined' error, meaning that `numPromise` is not a Promise. This is key. You need to play around with the `async.mapLimit(...)` expression until you hit on the right formulation to make it return Promise. — Roamer-1888, Aug 27 '19 at 00:15
It may be a version thing. Make sure you are using up to date `Async`. — Roamer-1888, Aug 27 '19 at 00:17
Tried few more things with 'async', but doesn't work. Can't even get the async/await (as suggested in documentation) running ;-) Btw., version is latest, so that should be fine. Thx for all hints though, definitely helped in getting me on track! @Roamer-1888 — Erik Reder, Aug 27 '19 at 08:38

score 7 · Accepted Answer · answered Aug 27 '19 at 16:25

7

Here's the probably better answer, though the other alternative seemed fine as well:

const async = require('async');
const delay = require('util').promisify(setTimeout);
const numPromise = async.mapLimit(['1','2','3','4','5'], 3, async num => delay(200).then(() => num*2))
// or const numPromise = async.mapLimit(['1','2','3','4','5'], 3, async num => {
//    await delay(200);
//    return num*2;
// })
numPromise.then(console.log)
// or numPromise.then((results) => console.log(results))

Please also see this issue thread on GitHub as reference: GitHub: async.mapLimit does not return a Promise

I believe the issue with my code was in the iteratee function - either initially it wasn't asynchronous or using 'callback' instead of 'return' I assume (or maybe both ;-) ).

answered Aug 27 '19 at 16:25

Erik Reder

303
3
14

It would save a lot of time and trouble if the documentation included an example or two. Hopefully this will help othe people. – Roamer-1888 Aug 27 '19 at 21:47
1

In the issue linked I also received confirmation that it's a bug. So hopefully this might get fixed in an upcoming version. There's also a thread about examples, so looking forward to seeing some improvements on that matter, as I really appreciate the async library. – Erik Reder Aug 28 '19 at 07:20
Good, you got acknowledgement that there's a bug. It's hard to imagine how `.mapLimit()` is coded to make `async` keyword required of the iteratee. If it were correctly coded a promise-returning normal `Function` should behave the same. Can only think there's a test for `AsyncFunction` in there somewhere. Bizarre! – Roamer-1888 Aug 28 '19 at 13:16

Erik Reder · Answer 2 · 2019-08-27T08:37:23.777

The solutions was astonishingly simple - while I still believe the code above should be working (according to documentation), I found that there is another package that was built to work with promises: 'promise-async'

So I replaced the line

async = require('async');

with

async = require('promise-async');

and now the code above is working as expected:

let numPromise = async.mapLimit(['1','2','3','4','5'], 3, function(num, callback){
    setTimeout(function(){
        num = num * 2,
        console.log(num);
        callback(null, num);
    }, 
    4000);
})
//Promise.all([numPromise]) //Promise.all is not needed
numPromise
.then((result) => console.log("success:" + result))
.catch(() => console.log("no success"));

Returns

2
4
6
8
10
success:2,4,6,8,10

Thanks to @Roamer-1888 for all hints, e.g. keeping callback in Iteratee and not needing Promise.all (which was more of a safety line anyway ;-) ) — Erik Reder, Aug 27 '19 at 08:39

async.mapLimit with Promise

2 Answers2

Linked