1

I have 30k+ active devices that connect through TCP sticky session from my application. Each request than creates its own threads and start receiving and sending data. Each requests around take 20 mins to send the complete packets over the network. It means that application need to create 30K threads approximately.

Problem is that my application sometimes become unresponsive and doesn't send and receive messages and it has happening randomly. I was reading on HERE that the max limit of threads in each process is 2000. I am wondering if this is the bad design to hold the 30k+ session and wait for new message to send and receive? What can be done to manage such load?

My Assumption/Suggestion:

1: Creating 2000 threads at max would cause the application to slow because of context switching? 2: At any point in time 28000 devices will be on wait because of worker thread limit?

System Information:

  • 64 bit
  • 8 CPU cores
shujaat siddiqui
  • 1,527
  • 1
  • 20
  • 41
  • You know that a 8 CPU core can only do 8 threads, with hyper threading 16 threads, so creating 30K threads with a hardware limitatin of max 16 threads seems odd to say the least. – Rand Random Oct 26 '21 at 21:40
  • 3
    Why even create threads at all TCP is I/O bound and doesn't need threads to run. - https://stackoverflow.com/questions/868568/what-do-the-terms-cpu-bound-and-i-o-bound-mean – Rand Random Oct 26 '21 at 21:42
  • Are you using TPL / async/await, if not you should IMHO. – Rand Random Oct 26 '21 at 21:43
  • @RandRandom even if I use TPL/asycn/await max working threads can be 2000 correct? still 28000 would be in waiting until the transfer completes. – shujaat siddiqui Oct 26 '21 at 21:51
  • No, because you will be limited by the max TCP connections for asp.net core. first google hit: https://stackoverflow.com/questions/58033308/how-to-manage-large-number-of-tcp-connections-using-asp-net-and-c-sharp – Rand Random Oct 26 '21 at 21:53
  • ok sure let me check. As far as why we creating threads is because we have a queue where data is saved against the device id. So in order to transfer packets simultaneously each thread reads the data against the device id and uses tcp connection to send that packet. – shujaat siddiqui Oct 26 '21 at 21:56
  • As far as tcp connections are concerned one can create upto 28000 tcp connections based on ephemeral ports limits of the operating system. @RandRandom – shujaat siddiqui Oct 26 '21 at 21:59
  • With TPL and `async` [there might be only one thread](https://blog.stephencleary.com/2013/11/there-is-no-thread.html). TCP you can have 16383 ephemeral ports *per IP address* – Charlieface Oct 26 '21 at 21:59
  • Never had the need to manually create threads on an asp.net server, so I can't help with that. All I can say is, you would only need to use TPL and let asp.net work its magic to handle the load. – Rand Random Oct 26 '21 at 22:00
  • @Charlieface agree if IP address is same but can contain as many as possible if IP address is different depends upon system resources. – shujaat siddiqui Oct 26 '21 at 22:04
  • @RandRandom thanks for suggestion I would look into it. – shujaat siddiqui Oct 26 '21 at 22:05
  • If you need further help, you would need to share some code. Your explantion so far, atleast for me, wasn't really precise. For example your comment with `As far as why we creating threads...` just sounds to me like it could be solved by simply using https://learn.microsoft.com/en-us/dotnet/api/system.collections.concurrent.concurrentqueue-1?view=net-5.0 - but I am pretty sure I am missing the greater picture. – Rand Random Oct 26 '21 at 22:11
  • Request per thread here is the problem. I have run huge TCP servers before (much bigger than this): async all the way. You're using it right now, in fact. Kestrel can host this trivially. – Marc Gravell Oct 27 '21 at 01:00
  • @Rand the question you linked to above seems to be talking about client connections *from* asp.net; for server connections, there isn't an enforced cap that I've found - and I've run servers with over 70k established inbound connections per process (split over a handful of ports, but not many) – Marc Gravell Oct 28 '21 at 09:24
  • @MarcGravell - didn't even read into the linked question, thats why the "disclaimer" first google hit, as I didn't bother to check if it is any good, but thanks for clarifing that it may be misleading – Rand Random Oct 28 '21 at 10:10

1 Answers1

3

Use fewer threads. Use async IO instead. I would strongly recommend using Kestrel to host this using the "pipelines" model, which would deal with threading, buffer management, and most of the TCP nuances for you, so you can just worry about your protocol and application code. It scales far beyond your needs (although for 30k sockets you might want to start using multiple ports, especially if you're behind a load balancer, due to ephemeral port exhaustion). Kestrel would require .NET Core or above, but the same approach is available on .NET Framework via Pipelines.Sockets.Unofficial. One guide to pipelines and Kestrel in this context is in a 4-part series here - other guides are available.

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900