1

I am new to NodeJS, and would like to know what is the best pattern to use to achieve the following.

An incoming REST API call requests a batch operation (say 100 items) to be performed.

  1. Express handler then has to make 100 external async API calls, which can take upto 120 seconds to return, but many minutes to complete.

  2. The handler just returns with an ACK, and then queues up the requests for processing. (Say in a database or memory)

I then need to process each request by making external API calls in parallell :-

(A) Each API call will return the number of "free resources" available (after the current API call is serviced) for additional parallel calls.

(B) However, these API calls need to be made in parallel, and should not be attempted if the number of "free resources" on the backend is less than a defined threshold.

The idea is simple :- The application should not make too many parallel API calls. It should attempt to leave at least say five backend resources free.

However as the number of free resources is only available after the first API call is made, its OK to go over the limit initially, and then scale back later.

In Java I would do this by:

  1. For the incoming batch populate queue with 100 request objects. (after storing state in DB)

  2. Having a fixed number of worker threads servicing the queue.

  3. Each worker thread would check a global object which would maintain count of free resources.

  4. If free resources are greater then threshold, make the API call, and update the global counter. (Also update DB to indicate API call made for request batch item. The item would be re-submitted to the queue if the API call failed for any reason)

I am not too concerned with optimising the number of threads running or initial spikes etc, as long as there is a decent attempt to not flood the backend with too many requests.

Any suggestions for any async / thread pool framework I can use?

Ahmed Ashour
  • 5,179
  • 10
  • 35
  • 56
Shahed
  • 111
  • 9
  • Is there any possibility the client itself can make the external API calls securely? If not (say it's to a backend database that requires administrative credentials), are you sure there's nothing you can do to optimize your query into 1 call, or just fewer? The reason I'm asking is because this does not sound like a very scalable server architecture, as your HTTP server and your backend API should ideally be running in separate processes to avoid becoming unresponsive to HTTP requests. Asynchronicity can only do so much to maximize performance, you also have to be aware of sheer volume. – Patrick Roberts Feb 19 '18 at 16:40
  • It's a bit hard to tell exactly what you're asking here. Stackoverflow does not do well with broad architectural questions that cover dozens of topics (and those kind of questions are generally off-topic too). That's why you already have one close vote as "too broad". Stack overflow does do well with specific questions like "how do I control how many requests to an external service are in flight at once?". I think you really need to focus this question on one part of your problem, not the overall system solution. – jfriend00 Feb 19 '18 at 16:44
  • A couple things to read: [Make several requests to an api that can only handle 20 requests a minute](https://stackoverflow.com/questions/33378923/make-several-requests-to-an-api-that-can-only-handle-20-request-a-minute/33379149#33379149) and [Promise.all() consumes all my RAM](https://stackoverflow.com/questions/46654265/promise-all-consumes-all-my-ram/46654592#46654592) and [Choose proper async method for batch processing for max requests per sec](https://stackoverflow.com/questions/36730745/choose-proper-async-method-for-batch-processing-for-max-requests-sec/36736593#36736593). – jfriend00 Feb 19 '18 at 16:54
  • Yes the question seems to be very broad. But what I am really looking for is suggestions on what is the equivalent in NodeJS to Java thread pools / queues – Shahed Feb 19 '18 at 18:53

0 Answers0