How to speed up or Increase efficiency in a list of api calls?

Question

I am having an array of ids like this


[
  "b1a9564e-4e99-41aa-9268-c620ccb0ea9f",
  "5d963ccf-a5da-4ac8-9c32-95e828db072d",
  "1cdbd1bd-6f32-44a2-9571-d5c1282e203e",
  "020d6d09-1971-42ab-a52d-d108e2d04515",
  "f2c4fe0c-70ff-43c1-b8c1-411f50a1fc5e",
  "e8d34d93-864a-4625-93bb-8cddf9970303",
  "f0eeceaf-f0f0-48d6-afc4-14bfb41edef0",
  "67eed6b0-aee4-407c-ab40-d7356a153176"
   .
   .
   .
   .
]

This is my method which takes List of ids as a parameter and calls API for each id in list

        public async Task<List<string>> GetListFHALoanDetails(List<string> loanGuids)
        {
            List<LoanFHADetails> loanFHADetails = new List<LoanFHADetails>();
            List<string> loanDetails = new List<string>();
            for (int i = 0; i < loanGuids.Count; i++)
            {
                LoanFHADetails tempDetails = new LoanFHADetails();
                tempDetails = await GetFHALoanDetails(loanGuids[I]); // Calling API Here


                if (tempDetails != null)
                {
                    loanDetails.Add(JsonConvert.SerializeObject(tempDetails));
                }
            }

            return loanDetails;
        }

The problem in the list may have many ids nearly 20k loans it taking so much time process the entire list. How to speed up the process?

You can break the list at client end in small chunks and then call API in multiple threads with those chunks. — Chetan, Jun 24 '20 at 11:24
If you have no control of the server to make it operate on batches, you could consider parallelising the API calls: https://stackoverflow.com/a/11565531/1064169 — Martin Costello, Jun 24 '20 at 11:24

Arcord · Answer 1 · 2020-06-24T11:44:10.953

You can try something like this to parallelize calls:

List<Task> tasks = new List<Task>();
for (int i = 0; i < loanGuids.Count; i++)
{
   tasks.Add(GetFHALoanDetails(loanGuids[I])); // Calling API Here
}

await Task.WhenAll(tasks);

In that case all calls will be sent in parallel (not waiting the previous one to start) and after the last line you are sure that you get all the answers.

You should maybe check with the API provider how much call you can do in parralel, there is maybe some limit.

Edit

Obviously as Nazariy Perepichka stated you should use an endpoint that allow you to get multiple item at the same time. Or consider to add it to the other API if you can modify it.

Edit2

Just realized you have 20k of elements. So you should really avoid to send all of them in parallel, you could maybe try to do 20 elements in parralel each time.

score -2 · Answer 2 · answered Jun 24 '20 at 11:24

Yes, and it is quite simple. Learn how to use async code and writ the code in such a way that you submit multiple requests in parallel. Deepeding on the API (http i.e. is limited to 4) that should allow you to significantly cut down the time - mostly beause right now you are going one by one and even the transfer time may kill you.

Out side that - not a lot you can do as you are the end user, not the person who is writing the API.

How to speed up or Increase efficiency in a list of api calls?

2 Answers2