2

I've written a network server application from scratch, so no IIS or Windows Form. I have been using Task.Run to hand off control over the life of a network session to run in the background.

Something like...

while (true)
{
    var tcpClient = tcpListener.Accept...
    Task.Run(() => ProcessSession(tcpClient));
}

...and then...

public async Task ProcessSession(TcpClient tcpClient)
{
    ...
    await tcpClient.GetStream().ReadAsync(...)
    ...
}

My hope was that when I did async / await with the network I/O in ProcessSession that the worker thread would get freed up until the I/O completed.

I'm finding that my server software is getting bogged down with only a few hundred connections, to the point where clients timeout trying to connect, or if they get connected that they have really slow network throughput. I was hoping to handle many thousands of connections per server.

When I look at ThreadPool.Get(Max/Available)Threads, it doesn't look like much is going on there. Process.GetCurrentProcess().Threads.Count is only in the low dozens, so it's not like it's a thread per connection. It's just bogged down.

Michael Balloni
  • 339
  • 2
  • 4
  • 9

2 Answers2

1

Leveraging the networking stack's built-in asynchrony is the right solution, and it looks like that's what you're doing with the TcpClient.

Given that you're "getting bogged down," investigate where the bottleneck is. No one can solve a performance problem without knowing what the problem is. What's your memory space look like? Where is time being spent? What does perfmon say is happening in your server process?

Greg D
  • 43,259
  • 14
  • 84
  • 117
  • So do you think that Task.Run with I/O-bound code is an okay way to go? Is there any other way to fire off I/O-bound code so that async / await works and doesn't cause a thread-per-connection? – Michael Balloni Feb 03 '15 at 20:56
  • Looking at Task Manager, there's a large amount of memory being used the box, 6.5 GB, of which the network server is using 3.5 GB. – Michael Balloni Feb 03 '15 at 21:11
  • Looking at perfmon, it's allocating in the 10's of millions of bytes / second. The Gen 0 heap is 280 MB. The Gen 1 heap is 300 MB. The Gen 2 heap is 750 MB. The Large Object Heap is 200 GB. For the network buffers, I use a ConcurrentQueue of MemoryStreams so that GC's not running all the time dealing with the constant flow of network bytes in and out. – Michael Balloni Feb 03 '15 at 21:14
  • I'm going to assume the LOH size is a typo that's supposed to be 200MB. Even so, is that volume of memory across the heaps what you expect to have in your application for the number of connections it's running? If this is a 32-bit process, you're applying significant pressure to your memory space. – Greg D Feb 03 '15 at 22:15
  • It can also be useful to more clearly define what "bogging down" is. Is the request latency that you're facing getting high? What's the acceptable threshold? How many connections does it take to surpass that threshold? Is it associated with another characteristic of the server (E.g., consumed memory crossing a particular boundary)? – Greg D Feb 03 '15 at 22:17
  • IE, measure measure measure :) – Greg D Feb 03 '15 at 22:20
  • Yeah, that LOH must have been a typo! I'm not sure what to expect for the memory sizes...it's a new server get its first taste of production traffic. We've got 12 GB of RAM on a 64-bit box, and we're using 6 GB overall, so as long as .NET can manage its 3.5 GB I think the box overall is in good shape. With bogged down, it means time-to-first-byte and network throughput. We're not happy with either, they're both off 10X of where we'd like them, and we haven't even gotten near full capacity. Also, CPU usage is low, down in the 10's and 20's. – Michael Balloni Feb 04 '15 at 01:03
  • How are your measurements looking? Have you instrumented your flow enough to know where the delay is happening? – Greg D Feb 04 '15 at 01:52
1

If ProcessSession does indeed not block then you can just call it:

while (true)
{
    var tcpClient = tcpListener.Accept...
    ProcessSession(tcpClient);
}

And throw away the resulting task. Be sure to log all errors.

Change the name to ProcessSessionAsync.

usr
  • 168,620
  • 35
  • 240
  • 369
  • Is there any downside to just firing this stuff off from otherwise synchronous code? Where will the await bubble up to? Nothing to worry about? – Michael Balloni Feb 03 '15 at 21:31
  • Task.Run was also a way to fire off stuff from otherwise asynchronous code. So that's not a problem by itself. The way I did it here has only one downside and that is if ProcessSession does block although it shouldn't, or if it runs significant compute before going async, accepting new clients will be delayed. It is therefore safer to use Task.Run at the expense of a small amount of overhead.; Not sure what you mean by "bubble up". Errors are thrown away in both styles. You must take care to log them. – usr Feb 03 '15 at 21:35