How to loop many http requests with axios in node.js

Question

I have an array of users where each user has an IP address.

I have an API that I send an IP as a request and it returns a county code that belongs to this IP.

In order to get a country code to each user I need to send separate request to each user.

In my code I do async await but it takes about 10 seconds until I get all the responses, if I don't do the async await, I don’t get the country codes at all. My code:

async function getAllusers() {
  let allUsersData = await usersDao.getAllusers();

  for (let i = 0; i < allUsersData.length; i++) {
    let data = { ip: allUsersData[i].ip };
    let body = new URLSearchParams(data);
    await axios
      .post("http://myAPI", body)
      .then((res) => {
        allUsersData[i].countryCode = res.data.countryCode;
      });
  }

  return allUsersData;
}

So, what exactly are you asking for help with? It sounds like you do get the codes when you use the `await`. How many items are in the `allUsersData` array? — jfriend00, Nov 27 '21 at 20:05
The problem is that if I use the await it takes too much time for all the reponses (about 10 seconds) and this is because it waits for the response after every request I have about 40 users — Y F, Nov 27 '21 at 20:14

score 1 · Answer 1 · answered Nov 27 '21 at 20:20

You can use Promise.all to make all your requests once instead of making them one by one.

let requests = [];
for (let i = 0; i < allUsersData.length; i++) {
    let data = { ip: allUsersData[i].ip };
    let body = new URLSearchParams(data);
    requests.push(axios.post("http://myAPI", body)); // axios.post returns a Promise
  }
try {
    const results = await Promise.all(requests);
    // results now contains each request result in the same order
    // Your logic here...
}
catch (e) {
    // Handles errors
}

jfriend00 · Accepted Answer · 2021-11-27T20:37:31.560

0

If you're just trying to get all the results faster, you can request them in parallel and know when they are all done with Promise.all():

async function getAllusers() {
  let allUsersData = await usersDao.getAllusers();

  await Promise.all(allUsersData.map((userData, index) => {
      let body = new URLSearchParams({ip: userData.ip});
      return axios.post("http://myAPI", body).then((res) => {
          allUsersData[index].countryCode = res.data.countryCode;
      });
  }));
  return allUsersData;
}

Note, I would not recommend doing it this way if the allUsersData array is large (like more than 20 long) because you'll be raining a lot of requests on the target server and it may either impeded its performance or you may get rate limited or even refused service. In that case, you'd need to send N requests at a time (like perhaps 5) using code like this pMap() here or mapConcurrent() here.

edited Nov 27 '21 at 20:37

answered Nov 27 '21 at 20:15

jfriend00

683,504
96
985
979

I'd like to... but I am new in stack overflow... and I don't have enough reputation. – Y F Nov 29 '21 at 13:02
@YF - You always have enough reputation to click the checkmark on an answer to your own question. You need reputation for up/down arrows, but not for the checkmark. – jfriend00 Nov 29 '21 at 22:06
You are right, thanks. – Y F Nov 30 '21 at 17:27

How to loop many http requests with axios in node.js

2 Answers2

Linked