14

In C#, I am calling a public API, which has an API limit of 10 calls per second. API has multiple methods, different users can call different methods at a time, hence there are chances that "Rate Limit Reached" Exception may occur.

I have the following class structure:

public class MyServiceManager
{
    public int Method1()
    {
    }

    public void Method2()
    {
    }

    public string Method3()
    {
    }
}

Multiple users can call different methods at a time, How can I maintain a static calling Queue or Task so that I can monitor all requests and entertain only 10 requests in a one second

Paul Karam
  • 4,052
  • 8
  • 30
  • 53
Asad Khan
  • 347
  • 4
  • 10
  • What type of data are you calling? Is it a static list of data that never changes and you can cache it? Is the 10 calls per second only the limit to free access to their API? – krillgar Jun 19 '17 at 11:30
  • Possible duplicate of [How to limit the number of active Tasks running via the Parallel Task Library?](https://stackoverflow.com/questions/17674184/how-to-limit-the-number-of-active-tasks-running-via-the-parallel-task-library). There may be better duplicates, or none at all (since this is about rate limiting, not number of tasks per se). – Jeroen Mostert Jun 19 '17 at 11:37
  • 1
    You can use a [SemaphoreSlim](https://msdn.microsoft.com/library/system.threading.semaphoreslim(v=vs.110).aspx) with 10 "slots" and release each "slot" after one second – Sir Rufo Jun 19 '17 at 11:37
  • There's a structural problem here in that, if you can't apply *any* rate limiting at the consumer side, you may end up with a lot of waiting tasks, possibly enough to eventually have you run out of resources. This is true regardless of whether you're using tasks or threads or an internal queue. If consumers will be "reasonable", you can make them wait. – Jeroen Mostert Jun 19 '17 at 11:42
  • Might want to take a look at this rate-limiter library - even if you roll your own it could be useful seeing how others have implemented it https://github.com/David-Desmaisons/RateLimiter – Justin Jun 19 '17 at 11:52
  • @SirRufo I have multiple methods in a class which might be accessed simultaneously, how can I implement SemaphoreSlim in each method. I want a common solution for my class which can apply check on every request – Asad Khan Jun 19 '17 at 12:08
  • Don't such throttle mechanisms often just deny the 11th API call? In other words, you may be well within your right to simply bomb out and return an HTTP 429 to the user, rather than be a nice guy and allow requests to pile up. Because in truth, queuing them up yourself opens to you a DDoS attack. So, don't do that. Keep track of the (UTC) datetimes of your last 10 requests, and if the 10th is less than a second ago, give a 429. If not, pop the oldest one off, add the new request's datetime. – Craig Brunetti Mar 26 '18 at 20:37

1 Answers1

21

You can build a TaskLimiter based on SemaphoreSlim

public class TaskLimiter
{
    private readonly TimeSpan _timespan;
    private readonly SemaphoreSlim _semaphore;

    public TaskLimiter(int count, TimeSpan timespan)
    {
        _semaphore = new SemaphoreSlim(count, count);
        _timespan = timespan;
    }

    public async Task LimitAsync(Func<Task> taskFactory)
    {
        await _semaphore.WaitAsync().ConfigureAwait(false);
        var task = taskFactory();
        task.ContinueWith(async e =>
        {
            await Task.Delay(_timespan);
            _semaphore.Release(1);
        });
        await task;
    }

    public async Task<T> LimitAsync<T>(Func<Task<T>> taskFactory)
    {
        await _semaphore.WaitAsync().ConfigureAwait(false);
        var task = taskFactory();
        task.ContinueWith(async e =>
        {
            await Task.Delay(_timespan);
            _semaphore.Release(1);
        });
        return await task;
    }
}

It will

  • wait for a semaphore "slot"
  • start the real task
  • release the semaphore slot after a given timespan when the real task has finished

Here a sample usage

public class Program
{
    public static void Main()
    {
        RunAsync().Wait();
    }

    public static async Task RunAsync()
    {
        var limiter = new TaskLimiter(10, TimeSpan.FromSeconds(1));

        // create 100 tasks 
        var tasks = Enumerable.Range(1, 100)
           .Select(e => limiter.LimitAsync(() => DoSomeActionAsync(e)));
        // wait unitl all 100 tasks are completed
        await Task.WhenAll(tasks).ConfigureAwait(false);
    }

    static readonly Random _rng = new Random();

    public static async Task DoSomeActionAsync(int i)
    {
        await Task.Delay(150 + _rng.Next(150)).ConfigureAwait(false);
        Console.WriteLine("Completed Action {0}", i);
    }

}
Sir Rufo
  • 18,395
  • 2
  • 39
  • 73
  • 1
    Fantastic answer. Not only did it provide a solution, your answer connected several previously disparate coding concepts I had floating around. Thanks for the contribution. – IdusOrtus Apr 19 '21 at 18:40
  • I have posted a similar `RateLimiter` class [here](https://stackoverflow.com/questions/65825673/partition-how-to-add-a-wait-after-every-partition/65829971#65829971), based on the same idea, where the drawbacks of using a `SemaphoreSlim` and fire-and-forget tasks are discussed. – Theodor Zoulias Jun 05 '23 at 20:54
  • Btw this use of `ContinueWith` violates the guideline [CA2008](https://learn.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca2008) about not creating tasks without passing a `TaskScheduler`. – Theodor Zoulias Jun 05 '23 at 20:56