2

I have a .NET web application, which for each request sends requests to one of the api from a set of multiple API's (which api to hit depends on the request type), gets the response, processes it and returns the response.

A brief overview of the architecture

Suppose at any given point of time, lets say we have a maximum of 100 threads, and we get 100 requests (lets assume each thread is handling one request each) for which the request needs to go to API-1 and suddenly API-1's response time increases, and then we get subsequent requests which need to go to API-2,3 ... n and all these API(s) are working perfectly.Requests for these API(s) won't be served until one the threads get free from processing API-1 which results in an overall performance impact of the .NET web application.

What I want to achieve is that I want to limit the number of threads for each API (lets say we have a class for each api having some methods) such that each class does not exceed the maximum number of threads allocated to it.

(If I have n classes and 100 threads , I should be able to divide them into thread pools of 100/n each)

I tried a few links for thread pools but couldn't achieve the same I wanted.

shubhamr
  • 445
  • 9
  • 23
  • How are you creating these threads per se ? – TheGeneral Apr 19 '18 at 06:52
  • Does `API-i` perform mostly CPU bound or I/O bound work? – Leonid Vasilev Apr 19 '18 at 07:01
  • @TheGeneral For now, leaving it to IIS to handle it.Not done at app level yet – shubhamr Apr 19 '18 at 07:09
  • @LeonidVasilyev Mostly I/O bound compared to CPU bound. – shubhamr Apr 19 '18 at 07:09
  • 2
    This is a prime case for async I/O and [using HttpClient right](https://www.google.nl/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwjN74uN68XaAhUBIVAKHWumCg4QFggpMAA&url=https%3A%2F%2Faspnetmonsters.com%2F2016%2F08%2F2016-08-27-httpclientwrong%2F&usg=AOvVaw0a0Pklclg_wNrIMVuxMVZE). You should only need 1 thread per incoming request, and ASP.NET handles that. – H H Apr 19 '18 at 07:42
  • 1
    If you are using async and are waiting on external I/O there is no 'blocked thread' waiting on API-1 that's impacting performance. Please show some code. It's very rare that you can do better than the system managing threads or thread priorities. – Ian Mercer Apr 19 '18 at 07:42

1 Answers1

1

Probably your application is a good target for a asynchronous programming model which, if used correctly, eliminates the blocked threads downtime issue.

Asynchronous programming in C# is a very broad topic that has been discussed a lot. Check following resources:

If you really need to, you can still limit number of ThreadPool threads (workers) as usual. See Using TPL how do I set a max threadpool size discussion on StackOverflow.

Leonid Vasilev
  • 11,910
  • 4
  • 36
  • 50
  • 1
    Yes for the asymc, __No__ on messing around with the ThreadPool. – H H Apr 19 '18 at 07:43
  • 1
    @HenkHolterman can you elaborate on ThreadPool workers limit? – Leonid Vasilev Apr 19 '18 at 08:06
  • 1
    You should be very weary of meddling with those settings at all, and certainly in an ASP.NET application. ASP.NET does a lot of thread management, don't get in the way. – H H Apr 19 '18 at 08:41
  • What would be the reason to limit number of ThreadPool threads at all? Since each task can be scheduled on any free thread, keeping default number of threads usually gives the best performance. – FCin Apr 19 '18 at 09:35
  • 1
    @FCin probably one reason is to balance performance / address space and memory consumption. It is limited by default. See [ThreadPool max threads](https://stackoverflow.com/a/6000891/2594597) answer by Jon Skeet. – Leonid Vasilev Apr 19 '18 at 09:49