0

I have a Node.js app that eventually takes a list, sends an HTTP request for each list item and responses are then processed as a list. This list can have up to 3000 objects, each one with multiple subproperties.

Currently I'm using Bluebird's Promise.map function to create those requests:

Promise.map( // takes items and sends them one by one.
  items,
  item => otherClass.sendHTTPRequest(item.prop1, item.prop2) // sendHTTPRequest() returns the response.
  )
  .then(results => {
    // `results` is an array with a response for each item.
  });

But recently a fellow adviced me to use queuing in Redis with RSMQ to prevent performance issues.

I know from this answer that promises are just triggered/launched and responses are handled as they come, so the app won't be executing a lot of response handlers simultaneously, but I never worked with so many requests, so I'm not sure the server won't eventually explode.

Is there a problem if I make multiple requests using Promise.map to iterate over big lists?

Abel
  • 21
  • 5
  • 2
    Bluebirds map has a concurrency option, just use that.. You might have to play with the concurrency value,.. Eg, you might find executing 20 requests at the same time might be better than say executing 100.. etc. – Keith Feb 21 '18 at 14:46
  • You might be right, I didn't set a value to `concurrency`. Thanks! – Abel Feb 21 '18 at 14:56

0 Answers0