0

I'm trying to make several GET requests to an API, in parallel, but I'm getting an error ("Too many requests") when trying to do large volumes of requests (1600 items).

The following is a snippet of the code.

Call:

var metadataItemList = await GetAssetMetadataBulk(unitHashList_Unique);

Method:

private static async Task<List<MetadataModel>> GetAssetMetadataBulk(List<string> assetHashes)
        {
            List<MetadataModel> resultsList = new();
            int batchSize = 100;
            int batches = (int)Math.Ceiling((double)assetHashes.Count() / batchSize);

            for (int i = 0; i < batches; i++)
            {
                var currentAssets = assetHashes.Skip(i * batchSize).Take(batchSize);
                var tasks = currentAssets.Select(asset => EndpointProcessor<MetadataModel>.LoadAddress($"assets/{asset}"));
                resultsList.AddRange(await Task.WhenAll(tasks));
            }
            return resultsList;
        }

The method runs tasks in parallel in batches of 100, it works fine for small volumes of requests (<~300), but for greater amounts (~1000+), I get the aforementioned "Too many requests" error. I tried stepping through the code, and to my surprise, it worked when I manually stepped through it. But I need it to work automatically.

Is there any way to slow down requests, or a better way to somehow circumvent the error whilst maintaining relatively good performance?

The request does not return a "Retry-After" header, and I also don't know how I'd implement this in C#. Any input on what code to edit, or direction to a doc is much appreciated!

The following is the Class I'm using to send HTTP requests:

class EndpointProcessor<T>
{
    public static async Task<T> LoadAddress(string url)
    {
        using (HttpResponseMessage response = await ApiHelper.apiClient.GetAsync(url))
        {
            if (response.IsSuccessStatusCode)
            {
                T result = await response.Content.ReadAsAsync<T>();
                return result;
            }
            else
            {
                //Console.WriteLine("Error: {0} ({1})\nTrailingHeaders: {2}\n", response.StatusCode, response.ReasonPhrase, response.TrailingHeaders);
                throw new Exception(response.ReasonPhrase);
            }
        }
    }
}
ARK
  • 11
  • 3
  • 1
    [429 Too Many Requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429) – phuzi May 10 '21 at 08:55
  • https://stackoverflow.com/a/23367215/592958 – phuzi May 10 '21 at 08:56
  • Does this answer your question? [How to avoid HTTP error 429 (Too Many Requests) python](https://stackoverflow.com/questions/22786068/how-to-avoid-http-error-429-too-many-requests-python) – phuzi May 10 '21 at 08:57
  • 3
    Obviously, the API needs you to throttle your requests down to a certain throughput. First step would be to fnd out what the API allows at max and then find a way to throttle your app down to a load below that (maybe a little bit lower than the max as a safety cussion). – Fildor May 10 '21 at 09:02
  • 2
    ^^ An essential part in that is: what metrics does the API use. I mean, maybe it's "max average of X requests per minute" or maybe it's a "sliding window of X time, in which Y number of requests are allowed" ... you would need to accomodate for the respective metrics on your side, then. – Fildor May 10 '21 at 09:06
  • You are exceeding the default backlog number (see https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.tcplistener.start?view=net-5.0&force_isolation=true#System_Net_Sockets_TcpListener_Start_System_Int32_) Stepping through the code you most likely aren't getting as many parallel connections at the same time you would get running in normal mode. – jdweng May 10 '21 at 10:03
  • @phuzi That doesn't really highlight how I could limit my request rate, in C# specifically. I'm brand new to all this and haven't done any HTTP stuff on Python. – ARK May 10 '21 at 18:30
  • @Fildor The API states: `10 requests per second after a period of 5 second burst` I'm guessing it means 600 requests per minute. In which case I'm over by 1000. I'm not sure how I'd delay one thread task from executing following another. – ARK May 10 '21 at 18:33

1 Answers1

0

You can use a semaphore as a limiter for currently active threads. Add a field of Semaphore type to your API client and initialize it with a maximum count and an initial count of, say, 250 or what you determine as a safe maximum number of running requests. In your method ApiHelper.apiClient.GetAsync(), before making the real connection, try to acquire the semaphore, then release it after completing/failing the download. This will allow you enforce a maximum number of concurrently running requests.

V.Lorz
  • 293
  • 2
  • 8