0

So right now I'm trying to figure out how to solve this issue. I need to call an external resource (API) and pass to it an object collection.

So lets say, I have a sender like this:

public class Sender{

    /// Send an message to an external api
    public async Task Send(object[] objectCollection){
    
    }
}

To problem is, there is a throughput limit on the API side. For example (5 objects per second). So I need to implement some kind of a waiting mechanism on my side(throttler) that will work with concurrent method calls inside a single APP.

  1. Is there a .NET throttler already or do I have to implement it myself?
  2. How would you implement the 5 objects per second rule?

My idea was to simply use an semaphore but that would only limit the concurrency and not account for the time limit.

Thanks for suggestions.

DavidWaldo
  • 661
  • 6
  • 24
  • how long does it take to create a collection of 5 items? – fubo Feb 07 '22 at 11:55
  • The question is whether you are able to handle the accumulation of work items if you can't send that many. The correct method would be to use a queue, which is designed to hold an arbitrary number of items. You could then create a background thread in a number of ways which loops every e.g. 200mS to service the queue. You could crank your own, which would be less code but maybe not as resilient to failures, or you could bring in .Net queues like those based on RabbitMS – Luke Briner Feb 07 '22 at 11:55
  • fubo: average is 200-500ms – DavidWaldo Feb 07 '22 at 11:57
  • should that also be considered? `waitingtime = 5 seconds - time to send collection` – fubo Feb 07 '22 at 11:59
  • I would say it depends on many factors. 1) Do you know the actual rate? If such, you can set up waiting, chunk-ing yourself. If not: 2) Does the response provide details in case you hit the limit? For e.x: does it provide a 'wait for x-time' to try again? If such, get that 'x' and just wait it. 3) Also, are you implementing it as a background service or something? Else your request (http) might timeout while waiting for your *Send* mechanism etc etc. – Ergis Feb 07 '22 at 12:10
  • You can throttle using semaphore. https://stackoverflow.com/questions/10806951/how-to-limit-the-amount-of-concurrent-async-i-o-operations?noredirect=1&lq=1 – jira Feb 07 '22 at 12:26
  • Ergis, its a background service that pulls collections from a queue with serveral threads doing the work. (number of threads = concurrent connection limit on the API side) and sends the collections. I need the thread to wait a few ms if rate is exceeded. I do know the maximal rate. – DavidWaldo Feb 07 '22 at 12:48
  • Jira, semaphore would still throttle the concurrency not the time limit. Sender can achieve sending in hundreds of ms. With concurrency limit of 5, its still gonna exceed the rate limit. – DavidWaldo Feb 07 '22 at 12:50

1 Answers1

0

So in the end, I used window sliding technique.

DavidWaldo
  • 661
  • 6
  • 24