Loop over a list of ids and foreach id get data from api

Question

Loop over a list of ids and foreach id get data from api but, i can make only 40 requests per 10 seconds, i'm getting data from tbdb but i'm getting kicked by the api here is my code

Popular.findAll({ limit: 25 }).then((filmes) => {

for (let index = 0; index < filmes.length; index++) {
    const element = filmes[index];
    const id_filme = element.id_filme;

i get each movie id so i can make requests to the api with this function..

function fetchdata(id_filme) {
  setTimeout(function () {
    axios
      .all([
        axios.get(
          `https://api.themoviedb.org/3/movie/${id_filme}?api_key=${process.env.API_KEY}&language=pt-BR`
        ),
        axios.get(
          `https://api.themoviedb.org/3/movie/${id_filme}/videos?api_key=${process.env.API_KEY}&language=pt-BR`
        ),
      ])
      .then(
        axios.spread(function (detalhes, linksyt) {
          var filme = detalhes.data;
          var youtube = linksyt.data;
          // console.log(filme, youtube);
          var filmesLista = new Array();
          const title = filme.title;
          const filmeid = filme.id;
          if (Object.keys(filme.genres).length === 0) {
            var genre = filme.genres;
          } else {
            var genre = filme.genres[0].name;
          }
          const overview = filme.overview;
          const poster_path =
            "https://image.tmdb.org/t/p/w500" + filme.poster_path;
          const release_date = filme.release_date;
          const vote_average = parseFloat(filme.vote_average);
          const backdrop_path =
            "https://image.tmdb.org/t/p/original" + filme.backdrop_path;
          const imdb_id = filme.imdb_id;
          const runtime = filme.runtime;

          if (Object.keys(youtube.results).length === 0) {
            var trailer = "";
          } else {
            var trailer =
              "http://www.youtube.com/watch?v=" + youtube.results[0].key;
          }

          filmesLista.push([
            filmeid,
            title,
            genre,
            overview,
            poster_path,
            release_date,
            vote_average,
            backdrop_path,
            imdb_id,
            runtime,
            trailer,
          ]);
          console.log(filmesLista);

          filmesLista.forEach((i) => {
            const filmeid = i[0];
            const title = i[1];
            const genre = i[2];
            const overview = i[3];
            const poster_path = i[4];
            const release_date = i[5];
            const vote_average = i[6];
            const backdrop_path = i[7];
            const imdb_id = i[8];
            const runtime = i[9];
            const trailer = i[10];

            Detalhes.create({
              id_filme: `${filmeid}`,
              title: `${title}`,
              genre: `${genre}`,
              release_date: `${release_date}`,
              vote_average: `${vote_average}`,
              imdb_id: `${imdb_id}`,
              runtime: `${runtime}`,
              overview: `${overview}`,
              poster_path: `${poster_path}`,
              backdrop_path: `${backdrop_path}`,
              trailer: `${trailer}`,
            });
          });
        })
      );
    o++;
    if (o < 25) {
      fetchdata(id_filme);
    }
  }, 10000);
  console.log("INSERIDO NO BANCO");
}
fetchdata(id_filme);

the code works but, works so fast and i get kicked every time i run with more than 40 ids.. i'm strugling on just make 40 request every 10 secconds. anyone can help me please?

i want to loop over all the ids on my database and for every id i want to make a request to an api.. but i cant make more than 40 requests for every 10 secconds.. — Wilsntn, Jun 21 '22 at 01:58
In this code you are making enough of requests to make the api server think you are performing an DOS attack. If the api server is yours, make an api to which receives multiple ids and gives results as an array. Otherwise try to use Promises with limited number of requests. — Rohit Mahto, Jun 21 '22 at 02:02
Perhaps you want something like [`mapConcurrent()`](https://stackoverflow.com/questions/46654265/promise-all-consumes-all-my-ram/46654592#46654592) which lets you specify how many simultaneous requests are in flight at once. For example, you could make 1000 requests, with no more than 5 in flight at any time. — jfriend00, Jun 21 '22 at 05:09

Loop over a list of ids and foreach id get data from api

0 Answers0