5

Background

I usually write node.js script based on async.js to control the work flow. Sometimes I found that based on async.js, the code seems still a 'hell'. With multiple nest the code is not readable and tricky to maintain. I did some search here and found some useful resources - but most of them are general concepts. So I am going to ask a question. Any feedback would be appreciated.

My common code 

var request = require('request');
var async = require('async');

var array = [1, 2, 3, 4 ,5 ,6];
var url = 'http://www.google.com';
async.mapLimit(array, 3, function(number, callback) {
       request(url + number, function(error, res, body){
                  //do sth
                 callback();
       });
}, function(err){//do nothing});

Question

So this time I want to replace this kind of code by Promise. Could you please help? If the question is duplicated, please suggest some useful resource url. Thanks!

J.Lyu
  • 932
  • 7
  • 16
  • Where is `url` defined? Do you want to process only first three elements in `array` array? – guest271314 Sep 27 '16 at 07:18
  • @guest271314 Hey, thanks for your attention. Url can be any string, it's not important here. For your second question, do you really know about async.js? – J.Lyu Sep 27 '16 at 07:21
  • 1
    You can use `Promise` and `Promise.all()` method to resolve this issue – abdulbarik Sep 27 '16 at 07:22
  • Have not tried `.mapLimit` method. Is expected result for first three elements of `array` to be processed? – guest271314 Sep 27 '16 at 07:22
  • `Promise.all` is not the answer — it runs all the requests right away. `.mapLimit` runs at most 3 at the time. – Aleksei Zabrodskii Sep 27 '16 at 07:24
  • can you describe better what you want to accomplish with this code? It seems like you are calling http://www.google.com1, http://www.google.com2, http://www.google.com3 and then expect to do something on success. As suggested by abdulbarik, you can achieve this easily and natively with Promise.all – iagomr Sep 27 '16 at 07:25
  • Hey guys. mapLimit doesn't mean only run the first three items. As @elmigranto tells, it runs at most 3 at the time, but finally will run all 6 items – J.Lyu Sep 27 '16 at 07:32
  • 1
    @J.Lyu you either write your own concurrency limiting, or use alternative Promise implementation like Bluebird, it has concurrency option. With HTTP requests to same URL you can use node's Agent class to limit number of concurrent requests to the same host. – Aleksei Zabrodskii Sep 27 '16 at 07:33
  • @elmigranto Thanks. I will try. Let me check more answers and work out my own solution. – J.Lyu Sep 27 '16 at 07:41
  • 1
    I would advise you to consider using Bluebird powered [request-promise](https://www.npmjs.com/package/request-promise) or native ES6 promises powered [request-promise-native](https://github.com/request/request-promise-native) in the first place. – Redu Sep 27 '16 at 08:23

1 Answers1

5

You will need to Promisify request, which can be done with native promises, but since you're using NodeJS, it would be preferable if you used the Bluebird promise library, which is almost strictly superior to the native implementation.

Then you can do:

const Promise = require('bluebird');
const requestAsync = Promise.promisify(request); // This is what bluebird saves you
// requestAsync returns a promise instead of accepting a callback.

const array = [1, 2, 3, 4, 5, 6];

const requestSingle = number => requestAsync(`${url}${number}`);

Promise.map(array, requestSingle, {concurrency: 3});

Note that Promise.map() is a Bluebird functionality, and not native.

The closest you can easily get with native is:

const arrayOfRequestPromises = array.map(requestSingle);

Promise.all(arrayOfRequestPromises)
  .then(allResults => {
    // do something
  })
  .catch(err => {
    // handle errors
  });

But then you lose the maximum 3 concurrent requests option.

Madara's Ghost
  • 172,118
  • 50
  • 264
  • 308
  • Thanks for MadaraUchiha, i will try on my side then let you know if it works well. Btw thank @jfriend00 – J.Lyu Sep 27 '16 at 07:47
  • Why do you have `array.requestSingle`? – jfriend00 Sep 27 '16 at 07:48
  • @jfriend00 Typo, this is too early in the morning for me. – Madara's Ghost Sep 27 '16 at 07:49
  • How do I implement following logic: #1 after connecting to http://www.google.com1, fetch the html, then save to 1.html after connecting to http://www.google.com2, fetch the html, then save to 2.html ... #2 In my async.js code, i can set timeout before callback. In other word, I will have some seconds' delay during each 3 items running. Can this be implemented here? Thanks – J.Lyu Sep 27 '16 at 08:04
  • @J.Lyu It can (and look good too!), but that's a bit broad for an answer on Stack Overflow, you should look into the Promise primitive, and how it abstracts asynchronous values. Promises are highly composable, you can wait for the first of N promises (implementing a timeout), and wait for the result of all promises (creating a waiting point for synchronization), you can use `.then()` to create chains of Promises, and you can do all those things together. – Madara's Ghost Sep 27 '16 at 08:06
  • yes you are all right man. Give me some minutes to understand all of those. Thanks for your help. – J.Lyu Sep 27 '16 at 08:20
  • hi. I just tried following code (only shows the main part): const requestSingle = number => setTimeout(function () { requestAsync(`${url}${number}`). then(response => { console.log(number + ' is running'); }).catch(err => { console.log(err + ' - ERROR'); }) }, 5000); Promise.map(array, requestSingle, { concurrency: 3 }); The final output generated at the same time. Seems concurrency : 3 doesn't work. – J.Lyu Sep 27 '16 at 08:34