1

I have a windows forms app, on click of button I want to loop a list of integers and make http post calls in such a way that only 50 parallel requests can be made per second. This is because the target http endpoint only supports 50 requests per second. To achieve this here is my logic:

I have declared the SemaphoreSlim throttler as a global variable:

var throttler = new SemaphoreSlim(50);

In the button click event, I am looping over a list of items and making http calls:

        List<Task> lstTasks = new List<Task>();
        foreach (var item in lstItems)
        {
            lstTasks.Add(CallAsyncMtd(item));
        }

The method that makes the http calls is:

private async Task CallAsyncMtd(int item, IProgress<int> progress)
{
    await throttler.WaitAsync(); // here I am doing an async wait for the semaphore so that not more than 5 threads can run at the same time

    try
    {
        await Task.Delay(1000).ConfigureAwait(false); //simulate an api call, configure await is false so the remaining code will run on separate thread            
    }
    catch (Exception e)
    {
        //do some exception handling
    }

    //saveResponseToMySQL(response) //save the http response to mysql database synchronously
    progress.Report(1); //report progress
    
    await Task.Delay(1000 * 1); //wait for 1 second
    throttler.Release(); //release semaphore
}

I am waiting for 1 second (in a separate thread) after making the request. Is there any better way so that I can make 50 requests per second without having to wait an additional second as per above logic?

variable
  • 8,262
  • 9
  • 95
  • 215
  • Why are you waiting at all? – gjhommersom Aug 31 '21 at 11:06
  • 1
    The semaphore allows 50 thread to run at the same time. I am waiting so that no more than 50 requests get made in a second. For example - if I remove the 1 second wait, then within a second there can be a situation such that 50 requests are made, and say 1 of them is complete and the 51st request is made. I want to prevent that situation. – variable Aug 31 '21 at 11:08
  • Define 'per second'. Do you mean 'once you start a request, you can't start a 51st request until 1000 milliseconds have passed' (i.e. is the timing window based on the first request, or is it based on a clock based second window)? What _specifically_ is the rate limiting rule? Is only a single process kicking off these requests? How many total number of requests are you initiating? Have you considered passing multiple numbers to a single API call (rather than initiate multiple API calls)? – mjwills Aug 31 '21 at 11:39
  • The requirements documentation says that we are allowed to only make atmost 50 requests per second. So in a second there cannot be more than 50 requests. If we do make more than 50 then we have to pay a fee for all the requests. So I am trying to avoid the fee. – variable Aug 31 '21 at 11:43
  • `Interlocked.Increment` on every request. Every 45th request compare current time to 45 requests before. Add a Sleep if needed. I think if you do that before the `await` then it should do what you want. – mjwills Aug 31 '21 at 11:49
  • What you are searching for is called "rate limit". The `SemaphoreSlim` out of the box limits the concurrency, not the rate. You can find a `RateLimiter` class in [this](https://stackoverflow.com/questions/65825673/partition-how-to-add-a-wait-after-every-partition "Partition: How to add a wait after every partition") question. Btw for a robust implementation, the `Release` call should be positioned inside a `finally` block, as shown [here](https://stackoverflow.com/questions/24139084/semaphoreslim-waitasync-before-after-try-block). – Theodor Zoulias Aug 31 '21 at 12:20
  • @TheodorZoulias - Is there any problem with alternate approaches recommended by asaf92 or ToTosty in the answer section? – variable Aug 31 '21 at 12:39
  • variable these answers are working, but they are suboptimal. With asaf92's approach you can't know when exactly the `ApiCall()` completed, by just awaiting the `CallAsyncMtd` task, because it imposes a delay after the completion of the `ApiCall()`. With ToTosty's approach you can't start all 50 `ApiCall()` operations at the same time, and the rounding errors of the `Task.Delay` method are accumulated, resulting in imprecise control of the throttling. – Theodor Zoulias Aug 31 '21 at 17:54
  • In asf92's approach, code below `await ApiCall()` executes after the api response is returned and then adjusts the delay accordingly. Please can you help me understand why you think we can't know when exactly ApiCall() completed? – variable Sep 01 '21 at 07:34
  • variable you know exactly when `ApiCall()` completed, but only locally, inside the `CallAsyncMtd` method. Callers of the `CallAsyncMtd` are losing this information, because of the imposed delay. Unless you consider the `IProgress progress` a sufficient conveyor of this information. It might be, depending on your requirements, but in general I don't think it is. You can't `await` an `IProgress` notification as you can `await` a `Task`. – Theodor Zoulias Sep 01 '21 at 23:25

4 Answers4

1

Use a StopWatch at the begining of the function, than check how much time has elapsed at the end and only Delay for the necessary amount of time.

Something like this:

private async Task CallAsyncMtd(int item, IProgress<int> progress)
{
    await throttler.WaitAsync(); // here I am doing an async wait for the semaphore so that not more than 5 threads can run at the same time

    var watch = new StopWatch();
    watch.Start();
    try
    {
        await ApiCall(); //simulate an api call, configure await is false so the remaining code will run on separate thread            
    }
    catch (Exception e)
    {
        //do some exception handling
    }

    //saveResponseToMySQL(response) //save the http response to mysql database synchronously
    progress.Report(1); //report progress
    watch.Stop();
    var timeToWait = Math.Max(0, 1000 - watch.Elapsed.Milliseconds);
    await Task.Delay(timeToWait); //wait for necessary amount of time
    throttler.Release(); //release semaphore
}
asaf92
  • 1,557
  • 1
  • 19
  • 30
  • 1
    Look like an interesting approach. Will this work with configure await False, where-in the code following the http api call runs on separate thread? I mean - does stopwatch work across threads? – variable Aug 31 '21 at 11:46
  • 1
    I don't see a reason why not. It's a local variable inside an `async` function. You're never touching the same `StopWatch` from 2 different threads at once. – asaf92 Aug 31 '21 at 11:51
0

I would use System.Threading.Tasks.Dataflow for this. With an ActionBlock you can specify the parallism of your task, and if there are 50 task that sleep after doing one call there are only 50 calls per secound:

//define the ActionBlock to call CallAsyncMtd for each item
var block = new ActionBlock<int>(
    async item => { 
        await CallAsyncMtd(item, progress);
    },
    // Specify a maximum degree of parallelism.
    new ExecutionDataflowBlockOptions
    {
        MaxDegreeOfParallelism = 50
    });


//Define a BatchBlock and link it to a ActionBlock to generate Blocks of 50 items
var linkOptions = new DataflowLinkOptions { PropagateCompletion = true };


 var oneSecondBatchBlock = new BatchBlock<int>(buffSize);
 var oneSecondStatsFeedBlock = new ActionBlock<int[]>(
     async (int[] messages) => 
            {
                var watch = new Stopwatch();
                watch.Start();
                //Add to ActionBlock doing the call
                foreach (int item in messages) block.Post(item);


                watch.Stop();
                var timeToWait = Math.Max(0, 1000 - watch.Elapsed.Milliseconds);
                await Task.Delay(timeToWait);
                   
               });

 //Link Blocks
 oneSecondBatchBlock.LinkTo(oneSecondStatsFeedBlock, linkOptions);

       

        

// Add items to block
foreach (var item in lstItems)
{
     oneSecondBatchBlock.SendAsync(i).Wait();//Add Work
}
    
//wait for finish
oneSecondBatchBlock.Complete();            
await oneSecondBatchBlock.Completion;
block.Complete();
await block.Completion;

-- Edit --

It make more sense to use a BatchBlock to split the work into parts of 50 pieces.

Daniel W.
  • 938
  • 8
  • 21
0

you could not add a Task.delay into the CallAsyncMtd() method, try:

foreach (var item in lstItems)
{
    await Task.Delay(20);
    lstTasks.Add(CallAsyncMtd(item));
}
ToTosty
  • 36
  • 5
0

You could chain the delay and releasing the semaphore but not await it. This blocks the Semaphore for an additional second but instantly allows processing the result.

// Wait 1 second befor releasing the semaphore but do not block this request.
Task.Delay(1000 *1 ).ContinueWith(_ => throttler.Release());

If this causes to much of a delay you can combine it with this answer https://stackoverflow.com/a/68998020/9271844 to reduce the delay.

gjhommersom
  • 159
  • 1
  • 7
  • Notice that `ContinueWith` has some pitfalls in terms of synchronization context, so if you want to avoid it it's possible to refactor the calling method to start a function that `Delay`s and releases the semaphore, but instead of immediately `await`ing it you can process the results in the meantime. – asaf92 Aug 31 '21 at 12:04