0

In order to avoid the "Error 429 Too Many Requests" error in browser, how can I modify the below code so that no more than 10 calls per second are made?

This 10 calls per second rate limit is imposed by a 3rd-party API, over which we have no control.

Thanks in advance!

// Array of hundreds of URLS
const urls = allNames.map(
        (name) =>
`https://example.com/api/v3/results/${name}?limit=120&apikey=123`
    );

// TODO add throttle so as to not exceed 10 calls per second
const getStaticProps = async () => {
            Promise.allSettled(
            urls.map((url) =>
                fetch(url).then((resp) => resp.json())
            )
        ).then(console.log);
};
Lil Robots
  • 53
  • 1
  • 12
  • Any reason you can't just use paging and only call the data that you need rather than filling up an array with hundreds of pieces of data that you probably don't immediately need? – Andy May 08 '22 at 22:10
  • Well, all data are immediately required for rendering on page as tables, charts, etc. – Lil Robots May 08 '22 at 22:15
  • Try https://stackoverflow.com/questions/40639432/what-is-the-best-way-to-limit-concurrency-when-using-es6s-promise-all – Christian Tapia May 08 '22 at 22:25

2 Answers2

0

You can fetch ten and wait for one second.

// Array of hundreds of URLS
const urlsList = allNames.map(
        (name) =>
`https://example.com/api/v3/results/${name}?limit=120&apikey=123`
    );

// TODO add throttle so as to not exceed 10 calls per second
// Fuction that returns a Promise that resolves after given Milliseconds
const timer = ms => new Promise(res => setTimeout(res, ms))

for (let i = 0; i < urlsList.length; i+=10) {
  getStaticProps(urlsList.slice(i-10, i));
  await timer(1000); //It will make 10 calls and wait 1 seconds
} 

const getStaticProps = async (urls) => {
            Promise.allSettled(
            urls.map((url) =>
                fetch(url).then((resp) => resp.json())
            )
        ).then(console.log);
};
Raven
  • 138
  • 1
  • 7
  • Thanks, though I'm getting this error on the timer() row `Syntax error: Unexpected reserved word 'await'.` Any idea what to change? – Lil Robots May 08 '22 at 22:30
  • 1
    Put that code in an async function. – Andy May 08 '22 at 22:40
  • Yes you need a async function to wrap the code const wrapperFunction = async () => { // Code goes here } // Call function somewhere wrapperFunction (); – Raven May 08 '22 at 23:57
0

Hopefully this helps :)

const urls = allNames.map(
  (name) => `https://example.com/api/v3/results/${name}?limit=120&apikey=123`
);

// 1s === 1000ms
const delay = async (ms) => new Promise((resolve) => setTimeout(resolve, ms));

const isEven = (num) => num % 2 === 0;

const getStaticProps = async () => {
  const response = await Promise.allSettled(
    urls.map(async (url, index) => {
      if (index && isEven(index / 5)) await delay(1000);
      return await fetch(url).then((res) => res.json());
    })
  );
};
AndyDev__
  • 29
  • 1
  • 4
  • Unfortunately, this still gets the same "Error 429 Too Many Requests" error in browser, – Lil Robots May 08 '22 at 23:10
  • Well, it may happen. But you specifically asked for 10 request limit. Maybe try lower amount of requests, because API has some rate limitation. – AndyDev__ May 14 '22 at 15:35