-1

I'm using semaphore to avoid AggregateException (Task was cancelled) and using processors count as a maxCount.

int processorCount = Environment.ProcessorCount;
var semaphore = new SemaphoreSlim(maxCount: processorCount, initialCount: processorCount);

But in this case it's running slowly (I'm making 1000 of I/O requests, but number of requests could increase up to 10 000 so need to be sure that I will don't have error, and I'm reading data from Redis and it takes 4,5 seconds with all side work), should I use available threads from threadpool or what value is a good practice to use as a maxCount when making asynchronous I/O requests?

SValley_Dev
  • 638
  • 4
  • 8
  • 1
    What kind of Exceptions are you really trying to avoid? An AggregateException is just a group of different Exceptions. What is the underlying Exception of the AggregateException? – Peter Bruins May 17 '22 at 19:19
  • @PeterBruins **Task was cancelled** – SValley_Dev May 17 '22 at 19:19
  • 2
    Could you please describe in more detail what you're attempting to achieve and how you're using semaphore to do that? – Zdeněk Jelínek May 17 '22 at 19:20
  • 1
    What is the source of task cancellation? Are you using a cancellation token and it times out? Why? Or is that thrown from a library you are using? – Zdeněk Jelínek May 17 '22 at 19:21
  • I'm making calls to Redis and sometimes getting **Task was cancelled** exception and I want to avoid this task by decreasing number of parallel calls using Semaphore – SValley_Dev May 17 '22 at 19:21
  • @ZdeněkJelínek from library – SValley_Dev May 17 '22 at 19:22
  • 4
    Then the proper maxcount is whatever you find your redis is capable of processing. If you're using async/await (i.e. semaphore.WaitAsync) properly, you don't need to worry about thread pool. However, a backlog of work will be created waiting at the semaphore without any upper bound. As the work done to communicate with Redis does not consume threads (unless the library API you're using is synchronous/blocking), the threads will return to thread pool as soon as they suspend on IO work. – Zdeněk Jelínek May 17 '22 at 19:24
  • @ZdeněkJelínek what will be if I have 100 000 requests, there could be not enough threads and kind of over subscription? – SValley_Dev May 17 '22 at 19:26
  • That's true, but I believe you'll run out of threads/CPU power regardless of workload, wouldn't you? – Zdeněk Jelínek May 17 '22 at 19:28
  • @ZdeněkJelínek I don't think so, but I'm not sure what exactly do you mean – SValley_Dev May 17 '22 at 19:30
  • So what do you mean by 100k requests? If you want to perform 100k requests, let's say on a single thread, you take one request, send it to redis, register continuation once done, suspend, take another request... Then at some point resposes start coming and it is possible that they won't get served until the last request is sent. If you put a semaphore in the middle, the difference is that the suspension does not occur due to network but due to the semaphore so there is one more wake up once the responses start coming and are processed. So there should not be much of a difference. – Zdeněk Jelínek May 17 '22 at 19:38
  • That's really a simplification and probably not exactly precise, so the question is which part of this do you disagree with so that we can focus on that? – Zdeněk Jelínek May 17 '22 at 19:40
  • @ZdeněkJelínek while proceeding with async request I'm adding this to the list of tasks, and after that I'm doing this ```Task.WhenAll(tasks);```, but I'm afraid that If there would be a lot of request, processor will run up all resources, that's is why I'm setting maxCount limitation to don't run too much concurrently – SValley_Dev May 17 '22 at 19:47
  • @ZdeněkJelínek so what maxCount value would you suggest for example if there is 100 000 of requests? – SValley_Dev May 17 '22 at 19:48
  • 2
    Well the you shouldn't use Task.WhenAll but batch the requests and Task.WhenAll the individual batches. You don't need a semaphore at all for that. Or in other words, instead of using all your threads to queue requests, you would use all your threads to stop at the semaphore. If this is really your concern. So your next question is probably "how do I set the batch size?" and my answer is start with something less than CPU thread count and see where you end. – Zdeněk Jelínek May 17 '22 at 19:57
  • I never really run out of threads like this outside of doing blocking/CPU-bound work so I cannot really advise. But if you provided a real scenario and your system specs, someone else might be able to help you. – Zdeněk Jelínek May 17 '22 at 19:58
  • 1
    @ZdeněkJelínek OK, understand you, thank for discussing this with me :) – SValley_Dev May 17 '22 at 20:00
  • 2
    [Throttling asynchronous tasks](https://stackoverflow.com/q/22492383/7444103) -- [Queue of async tasks with throttling which supports muti-threading](https://stackoverflow.com/q/34315589/7444103) etc. – Jimi May 17 '22 at 20:34

1 Answers1

0

You do async work for I/O-bound work. Work that your code is "waiting" for, such as a response from Redis. In this "waiting" time you can do other work, like other async requests.

So, this means async work is not directly bound to your processor count (it is indirectly), because it can handle more async requests when "waiting". How much more work depends on the async work you are doing, hardware, latency, etc. The best is to experiment with how much you can handle per thread.

I usually start with maxCount=20, which is very safe. I often succeed in going to maxCount=50 without any problems.

Because it's still indirectly bound to processor threads, but you don't know how much work you can do per thread, I use a multiplier of processor counts.

So I use Environment.ProcessorCount * 4. Then I experiment in increasing the multiplier to 6 or 8.

Remy van Duijkeren
  • 9,894
  • 1
  • 19
  • 12