Here's an async and sync implementation of a throttle that can limit the number of calls to a method per duration of time. It's based on a simple comparison of the current time to DateTimeOffset and Task.Delay/Thread.Sleep. It should work fine for many implementations that don't need a high degree of time resolution, and should be called BEFORE the methods that you want to throttle.
This solution allows the user to specify the number of calls that are allowed per duration (with the default being 1 call per time period). This allows your throttle to be as "burstable" as you need at the cost of no control over when the callers can continue, or calls can be as evenly spaced as possible.
Let's say let’s say the target is 300 calls/min: you could have a regular throttle with a duration of 200ms that will evenly spread out every call with at least a minimum of 200ms in between, or you could create a throttle that will allow 5 calls every second with no regard to their spacing (first 5 calls win – might be all at once!). Both will keep the rate limit under 300calls/min, but the former is on the extreme end of evenly separated and the latter is more “bursty”. Having things evenly spread out is nice when processing items in a loop, but may not be so good for things running in parallel (like web requests) where the call times are unpredictable and unnecessary delays might actually slow down throughput. Again, your use case and testing will have to be your guide on which is best.
This class is thread-safe and you'll need to keep a reference to an instance of it somewhere that is accessible to the object instances that need to share it. For an ASP.NET web application that would be a field on the application instance, could be a static field on a web page/controller, injected from the DI container of your choice as a singleton, or any other way you could access the shared instance in your particular scenario.
EDIT: Updated to ensure the delay is never longer than the duration.
public class Throttle
{
/// <summary>
/// How maximum time to delay access.
/// </summary>
private readonly TimeSpan _duration;
/// <summary>
/// The next time to run.
/// </summary>
private DateTimeOffset _next = DateTimeOffset.MinValue;
/// <summary>
/// Synchronize access to the throttle gate.
/// </summary>
private readonly SemaphoreSlim _mutex = new SemaphoreSlim(1, 1);
/// <summary>
/// Number of allowed callers per time window.
/// </summary>
private readonly int _numAllowed = 1;
/// <summary>
/// The number of calls in the current time window.
/// </summary>
private int _count;
/// <summary>
/// The amount of time per window.
/// </summary>
public TimeSpan Duration => _duration;
/// <summary>
/// The number of calls per time period.
/// </summary>
public int Size => _numAllowed;
/// <summary>
/// Crates a Throttle that will allow one caller per duration.
/// </summary>
/// <param name="duration">The amount of time that must pass between calls.</param>
public Throttle(TimeSpan duration)
{
if (duration.Ticks <= 0)
throw new ArgumentOutOfRangeException(nameof(duration));
_duration = duration;
}
/// <summary>
/// Creates a Throttle that will allow the given number of callers per time period.
/// </summary>
/// <param name="num">The number of calls to allow per time period.</param>
/// <param name="per">The duration of the time period.</param>
public Throttle(int num, TimeSpan per)
{
if (num <= 0 || per.Ticks <= 0)
throw new ArgumentOutOfRangeException();
_numAllowed = num;
_duration = per;
}
/// <summary>
/// Returns a task that will complete when the caller may continue.
/// </summary>
/// <remarks>This method can be used to synchronize access to a resource at regular intervals
/// with no more frequency than specified by the duration,
/// and should be called BEFORE accessing the resource.</remarks>
/// <param name="cancellationToken">A cancellation token that may be used to abort the stop operation.</param>
/// <returns>The number of actors that have been allowed within the current time window.</returns>
public async Task<int> WaitAsync(CancellationToken cancellationToken = default(CancellationToken))
{
await _mutex.WaitAsync(cancellationToken)
.ConfigureAwait(false);
try
{
var delay = _next - DateTimeOffset.UtcNow;
// ensure delay is never longer than the duration
if (delay > _duration)
delay = _duration;
// continue immediately based on count
if (_count < _numAllowed)
{
_count++;
if (delay.Ticks <= 0) // past time window, reset
{
_next = DateTimeOffset.UtcNow.Add(_duration);
_count = 1;
}
return _count;
}
// over the allowed count within the window
if (delay.Ticks > 0)
{
// delay until the next window
await Task.Delay(delay, cancellationToken)
.ConfigureAwait(false);
}
_next = DateTimeOffset.UtcNow.Add(_duration);
_count = 1;
return _count;
}
finally
{
_mutex.Release();
}
}
/// <summary>
/// Returns a task that will complete when the caller may continue.
/// </summary>
/// <remarks>This method can be used to synchronize access to a resource at regular intervals
/// with no more frequency than specified by the duration,
/// and should be called BEFORE accessing the resource.</remarks>
/// <param name="cancellationToken">A cancellation token that may be used to abort the stop operation.</param>
/// <returns>The number of actors that have been allowed within the current time window.</returns>
public int Wait(CancellationToken cancellationToken = default(CancellationToken))
{
_mutex.Wait(cancellationToken);
try
{
var delay = _next - DateTimeOffset.UtcNow;
// ensure delay is never larger than the duration.
if (delay > _duration)
delay = _duration;
// continue immediately based on count
if (_count < _numAllowed)
{
_count++;
if (delay.Ticks <= 0) // past time window, reset
{
_next = DateTimeOffset.UtcNow.Add(_duration);
_count = 1;
}
return _count;
}
// over the allowed count within the window
if (delay.Ticks > 0)
{
// delay until the next window
Thread.Sleep(delay);
}
_next = DateTimeOffset.UtcNow.Add(_duration);
_count = 1;
return _count;
}
finally
{
_mutex.Release();
}
}
}
This sample shows how the throttle can be used synchronously in a loop, as well as how cancellation behaves. If you think of it like people getting in line for a ride, if the cancellation token is signaled it's as if the person steps out of line and the other people move forward.
var t = new Throttle(5, per: TimeSpan.FromSeconds(1));
var c = new CancellationTokenSource(TimeSpan.FromSeconds(22));
foreach(var i in Enumerable.Range(1,300)) {
var ct = i > 250
? default(CancellationToken)
: c.Token;
try
{
var n = await t.WaitAsync(ct).ConfigureAwait(false);
WriteLine($"{i}: [{n}] {DateTime.Now}");
}
catch (OperationCanceledException)
{
WriteLine($"{i}: Operation Canceled");
}
}
