How to make multiple API calls with a delay between each in Node.js

Question

My goal was to make multiple api calls for a list of data. Lets say I had the following code

const  axios = require('axios');

const  axiosRequests = [];
const strings = ['a', 'b', 'c'];
for (let  str  of  strings) {
    axiosRequests.push(axios.get(`https://www.apiexample.com/get/?cfg=json&value=${str}`))
}

The easiest solutions was to apply the following:

let  responseArray;
try {
    responseArray = await  Promise.all(axiosRequests);
} catch (err) {
    console.log(err);
}

responseArray.map(response  => {
    //make something with the response
{

But the problem I encountered with from the API was the HTTP 429 Too Many Requests response status code, which means that the API restricts the number of requests for a period of time.

I want to add a delay between each request.

How can I do this?

and how can I gather all of the responses to responses array? — Efi Gordon, May 03 '20 at 04:35
Does this answer your question? [multiple requests with for loop and timeout between each](https://stackoverflow.com/questions/53026219/multiple-requests-with-for-loop-and-timeout-between-each) — Charlie, May 03 '20 at 05:52
@EfiGordon, you just put the result of each awaited request into an array, and return that. — Evert, May 03 '20 at 05:59

xdeepakv · Accepted Answer · 2020-05-03T05:46:51.743

You can call in series. However, I recommend using chunks, to make it more useful.

Using chunk, best performance:

const delay = (ms = 1000) => new Promise((r) => setTimeout(r, ms));

const getInChunk = async function (items, chunkSize) {
  let results = [];
  let chunkPromises = [];
  let chunkResults = [];
  for (let index = 0; index < items.length; index++) {
    if (index % chunkPromises === 0) {
      chunkPromises = [];
      chunkResults.push(await Promise.all(chunkPromises));
    } else {
      chunkPromises.push(
        axios.get(`https://jsonplaceholder.typicode.com/todos/${items[index]}`).then(res => res.data)
      );
    }
  }
  // last chunk
  if (chunkPromises.length) {
    chunkResults.push(await Promise.all(chunkPromises));
  }
  // flatten 
  chunkResults.forEach(chunk =>{
    results = results.concat(chunk)
  })
  console.log(results)
  return results;
};

async function main() {
  const strings = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
  const results = await getInChunk(strings, 5);
  console.log(results);
}
main();

<script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.19.2/axios.min.js"></script>

Simple:

const axios = require("axios");
const delay = (ms = 1000) => new Promise((r) => setTimeout(r, ms));
const getInSeries = async (promises) => {
  let results = [];
  for (let promise of promises) {
    results.push(await delay().then(() => promise));
  }
  return results;
};
const getInParallel = async (promises) => Promise.all(promises);
async function main() {
  const strings = [1, 2, 3, 4];
  const promises = strings.map((id) =>
    axios
      .get(`https://jsonplaceholder.typicode.com/todos/${id}`)
      .then((res) => res.data)
  );
  const results = await getInSeries(promises);
  console.log(results);
}
main();

<script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.19.2/axios.min.js"></script>

Performance friendly series. loop one time O(N).

const delay = (ms = 1000) => new Promise((r) => setTimeout(r, ms));
const getTodosSeries = async function (items) {
  let results = [];
  for (let index = 0; index < items.length; index++) {
    await delay();
    const res = await axios.get(
      `https://jsonplaceholder.typicode.com/todos/${items[index]}`
    );
    results.push(res.data);
  }
  return results;
};

async function main() {
  const strings = [1, 2, 3, 4];
  const results = await getTodosSeries(strings);
  console.log(results);
}
main();

<script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.19.2/axios.min.js"></script>

Great answer! But how can I add a delay between each request? — Efi Gordon, May 03 '20 at 04:56
Thanks- I will use this case it is really great solution - Did you mean `if (index % chunkSize === 0)` (in the first version - Using chunk, best performance) — Efi Gordon, May 18 '20 at 08:28
chunk snippet: 1. "chunkSize" is not used 2. "index % chunkPromises" you divide number by an array 3. you use "await" inside a loop 4. you are not using a "delay()" function Conclusion: this code won't work as expected — Aleks Shenshin, Mar 01 '22 at 21:47

jfriend00 · Answer 2 · 2021-01-23T03:19:02.780

There are a variety of strategies for dealing with too many requests and which strategy gets you the best throughput without running afoul of the target server rate limiting depends entirely upon exactly how the target server is measuring and enforcing things. Unless that is documented, you would have to just experiment. The safest (and potentially slowest) strategy is to run your requests sequentially with a delay between them and tune that delay time as appropriate.

Run Sequentially With Delay Between Each Request

You can run your requests sequentially and use await on a delay promise to separate them in time.

const  axios = require('axios');

function delay(t) {
    return new Promise(resolve => setTimeout(resolve, t));
}

async function getResults() {

    const results = [];
    const strings = ['a', 'b', 'c'];
    for (let  str  of  strings) {
        await delay(1000);
        let data = await axios.get(`https://www.apiexample.com/get/?cfg=json&value=${str}`);
        results.push(data);
    }
    return results;
}

getResults().then(results => {
    console.log(results);
}).catch(err => {
    console.log(err);
});

Run N Requests at a Time where N > 1 and N < all your Requests

If you want to run N requests at a time where N is more than 1 (often like 3 or 4) but less than all your requests, then see mapConcurrent() in this answer. Whether this is feasible not as many as you were doing depends entirely upon the target server and what exactly it is measuring and enforcing.

Actual Rate Limiting where You Run N Requests per Second

For actual rate limiting where you control the requests per second directly, then see rateLimitMap() in this answer: Choose proper async method for batch processing for max requests/sec.

These seems like it should work...but I get `getResults.then is not a function` — Kyle Pennell, Jan 23 '21 at 03:17
@KylePennell - It's supposed to be `getResults().then(...)` which was a typo that I just fixed. — jfriend00, Jan 23 '21 at 03:19
https://www.apiexample.com/get/?cfg=json&value=d doesn't work (maybe gone/down)...perhaps you could put in jsonplaceholder But the idea is right — Kyle Pennell, Jan 23 '21 at 03:25
@KylePennell - That url is something the OP used in their question - I'm not responsible for it still working or not. And, this answer is from 7 months ago. You need to use your own URL. — jfriend00, Jan 23 '21 at 03:36

Charlie · Answer 3 · 2020-05-03T05:50:48.423

0

Go simple. Inject a delay into each request in your loop.

for (const [i, v] of ['a', 'b', 'c'].entries()) {
   setTimeout(()=>{
      axiosRequests.push(axios.get(`https://www.apiexample.com/get/?cfg=json&value=${str}`))}, 
      i * 100)
}

edited May 03 '20 at 05:50

answered May 03 '20 at 05:06

Charlie

22,886
11
59
90

Omar Omeiri · Answer 4 · 2022-03-21T20:49:35.980

For anyone that prefers object oriented style, creating a reusable decorator can be more readable.

function delayAsync(delayMs = 0): any {
  return function (target: any, propertyKey: string, descriptor: PropertyDescriptor) {
    const prevMethod = descriptor.value;
    descriptor.value = async function (...args: any[]) {
      await Promise.resolve(
        new Promise<void>((resolve) => {
          setTimeout(() => { resolve(); }, delayMs);
        }),
      );
      return prevMethod.apply(this, args);
    };
    return descriptor;
  };
}

class Test {
  @delayAsync(500)
  async test() {
    console.log('test');
  }
}

const test = new Test();
(async () => {
  for await (const i of Array(10).keys()) {
    await test.test();
  }
})();

You can test it here

How to make multiple API calls with a delay between each in Node.js

4 Answers4

Linked