0

I need to call an API thousands of times as quickly as possible. The API has a limit of 10 calls per second. In order to take full advantage of the 10 calls per second without going over, I'm calling the API asynchronously and throttling the calls with a semaphore and a timer. My code enters the semaphore, calls the API, and then makes sure at least one second has passed before it releases the semaphore.

The API call is actually pretty quick and returns in about a second or less, so my code should move right on to the check time/release semaphore logic. However, what actually happens is after 10 calls the semaphore fills up and is not released at all until the rest of the async Tasks to call the API are created. After that everything works as expected, so I'm not experiencing any real issues. The behavior just seems strange and I would like to understand it.

public static class MyClass 
{
    SemaphoreSlim semaphore = new SemaphoreSlim(10);

    public static async Task CreateCallApiTasks(IList<object> requests)
    {
        var callApiTasks = requests.Select(x => CallApi(x));
        await Task.WhenAll(callApiTasks);
    }

    private static async Task CallApi(object requestBody)
    {
        using (var request = new HttpRequestMessage(HttpMethod.Post, <apiUri>))
        {
            request.Content = new StringContent(requestBody, System.Text.Encoding.UTF8, "application/json");
            HttpResponseMessage response = null;

            using (var httpClient = new HttpClient())
            {
                var throttle = new Stopwatch();
                ExceptionDispatchInfo capturedException = null;

                await semaphore.WaitAsync();

                try
                {
                    throttle.Start();
                    response = await httpClient.SendAsync(request);

                    while (throttle.ElapsedMilliseconds < 1000)
                    {
                        await Task.Delay(1);
                    }

                    semaphore.Release();
                    throttle.Stop();
                }
                catch (Exception ex)
                {
                    capturedException = ExceptionDispatchInfo.Capture(ex);
                }

                if (capturedException != null)
                {
                    while (throttle.ElapsedMilliseconds < 1000)
                    {
                        await Task.Delay(1);
                    }

                    semaphore.Release();
                    throttle.Stop();                
                    capturedException.Throw();
                }
            }
        }
    }
}
jtrohde
  • 412
  • 2
  • 4
  • 16
  • 1
    You can see my implementation of a throttled queue [here](https://stackoverflow.com/a/34316994/1159478). – Servy Oct 26 '18 at 17:25
  • Your code has numerous compiler errors throughout, from things being the wrong types, to instance variables in a static class, etc. – Servy Oct 26 '18 at 17:27
  • 1
    The `Task` returned by `CallApi` in your case doesn't complete until the full second is up from when it started, rather than being completed as soon as the actual API call has finished. That's very likely to cause problems for the calling code. – Servy Oct 26 '18 at 17:30

0 Answers0