Node, wait and retry api calls that fail

Question

So I fetch an array of urls from api with a rate limit, currently I handle this by adding a timeout to each call like this:

const calls = urls.map((url, i) =>
  new Promise(resolve => setTimeout(resolve, 250 * i))
    .then(() => fetch(url)
  )
);

const data = await Promise.all(calls);

forcing a 250ms wait between each call. This ensures that the rate limit is never exceeded.

---

The thing is, this isn't really necessary. I've tried with 0ms wait time, and most of the cases I have to repeatedly reload the page four or five times before the api starts to return:

{ error: { status: 429, message: 'API rate limit exceeded' } }

and most of the times you only have to wait a second or so before you can safely reload the page and get all data.

A more reasonable approach would be to collect the calls that return 429 (if they do), wait for a set amount of time and then retry them (and perhaps redo this a set amount of times).

Problem, I'm a bit stumped as to how one would go about achieving this?

EDIT:
Just got home and will look through the answers but there seem to have been an assumption made which I don't believe is necessary: The calls does not have to be sequential, they can be fired (and returned) in any order.

Answer 1

The term for what you want is exponential backoff. You can modify your code so that it continues trying on a certain failure condition:

const max_wait = 2000;

async function wait(ms) {
  return new Promise(resolve => {
    setTimeout(resolve, ms);
  });
}

const calls = urls.map(async (url) => {
  let retry = 0, result;

  do {
    if (retry !== 0) { await wait(Math.pow(2, retry); }
    result = await fetch(url);
    retry++;
  } while(result.status !== 429 || (Math.pow(2, retry) > max_wait))

  return result;
}

Or you can try using a library to handle the backoff for you like https://github.com/MathieuTurcotte/node-backoff

Answer 2

Here is an example that allows to handle an array of promises sequencially, by setting a delay expressed in milliseconds and accepting a third callback determining whether the request should be retried.

In the below code, some sample requests are mocked to:

• Test a successful response.
• Test an error response. If the error response contains an error code and the error code is 403, true is returned and the call is retried in the next run (delayed by x milliseconds).
• Test an error response without an error code.

There is a global counter below that give up the promise after N tries (in the below example 5), all of that is handled in this code:

const result = await resolveSequencially(promiseTests, 250, (err) => {
    return ++errorCount, !!(err && err.error && err.error.status === 403 && errorCount <= 5);
  });

Where the error count is first increased and it returns true if the error is defined, has an error property and its status is 403. Of course, the example is just to test things out, but I think you're looking for something allowing you to have a cleverer control over the promise loop cycle, hence here is a solution doing just that.

I will add some comments below, you can run the test below to check what happens directly in the console.

// Nothing that relevant, this one is just for testing purposes!
let errorCount = 0;

// Declare the function.
const resolveSequencially = (promises, delay, onFailed, onFinished) => {
    // store the results.
    const results = [];
    // Define a self invoking recursiveHandle function.
   (recursiveHandle = (current, max) => { // current is the index of the currently looped promise, max is the maximum needed.
    console.log('recursiveHandle invoked, current is, ', current ,'max is', max);
    if (current === max) onFinished(results); // <-- if all the promises have been looped, resolve.
    else {
      // Define a method to handle the promise.
      let handlePromise = () => {
        console.log('about to handle promise');
        const p = promises[current];
        p.then((success) => {
          console.log('success invoked!');
          results.push(success);
          // if it's successfull, push the result and invoke the next element.
          recursiveHandle(current + 1, max);
        }).catch((err) => {
          console.log('An error was catched. Invoking callback to check whether I should retry! Error was: ', err);
          // otherwise, invoke the onFailed callback.
          const retry = onFailed(err);
          // if retry is true, invoke again the recursive function with the same indexes.
          console.log('retry is', retry);
          if (retry) recursiveHandle(current, max);
          else recursiveHandle(current + 1, max); // <-- otherwise, procede regularly.
        });
      };
      if (current !== 0) setTimeout(() => { handlePromise() }, delay); // <-- if it's not the first element, invoke the promise after the desired delay.
      else handlePromise(); // otherwise, invoke immediately.
    }
  })(0, promises.length); // Invoke the IIFE with a initial index 0, and a maximum index which is the length of the promise array.
}

const promiseTests = [
  Promise.resolve(true),
  Promise.reject({
    error: {
      status: 403  
    }
  }),
  Promise.resolve(true),
  Promise.reject(null)
];

const test = () => {
  console.log('about to invoke resolveSequencially');
  resolveSequencially(promiseTests, 250, (err) => {
    return ++errorCount, !!(err && err.error && err.error.status === 403 && errorCount <= 5);
  }, (done) => {
    console.log('finished! results are:', done);
  });
};
test();

Answer 3

If I understand the question right, your trying to:

a) Execute fetch() calls sequentially (with a possibly optional delay)

b) Retry failed requests with a backoff delay

As you likely found out, .map() does not really help with a) as it does not wait for any async stuff when iterating (which is why you create a greater and greater timeout with i*250).

I personally find it the easiest to keep things sequential by using a for of loop instead, as this will work nicely with async/await:

const fetchQueue = async (urls, delay = 0, retries = 0, maxRetries = 3) => {

 const wait = (timeout = 0) => {
  if (timeout) { console.log(`Waiting for ${timeout}`); }

  return new Promise(resolve => {
   setTimeout(resolve, timeout);
  });
 };

 for (url of urls) {
  try {
   await wait(retries ? retries * Math.max(delay, 1000) : delay);

   let response = await fetch(url);
   let data = await (
    response.headers.get('content-type').includes('json')
    ? response.json()
    : response.text()
   );
    
   response = {
    headers: [...response.headers].reduce((acc, header) => {
     return {...acc, [header[0]]: header[1]};
    }, {}),
    status: response.status,
    data: data,
   };

   // in reality, only do that for errors
   // that make sense to retry
   if ([404, 429].includes(response.status)) {
    throw new Error(`Status Code ${response.status}`);
   }

   console.log(response.data);
  } catch(err) {
   console.log('Error:', err.message);

   if (retries < maxRetries) {
    console.log(`Retry #${retries+1} ${url}`);
    await fetchQueue([url], delay, retries+1, maxRetries);
   } else {
    console.log(`Max retries reached for ${url}`);
   }
  }
 }
};

// populate some real URLs urls to fetch
// index 0 will generate an inexistent URL to test error behaviour
const urls = new Array(101).fill(null).map((x, i) => `https://jsonplaceholder.typicode.com/todos/${i}`);

// fetch urls one after another (sequentially)
// and delay each request by 250ms
fetchQueue(urls, 250);

If a request fails (e.g. you get one of the errors specified in the array with error status codes), the above function will retry them a maximum of 3 times (by default) with a backoff delay that increases by a second on each retry.

As you wrote, the delay between requests is probably not necessary, so you could just remove the 250 in the function call. Because each request is executed one after the other, you're less likely to run into rate limit issues but if you do, it's very easy to add some custom delay.