3

I've done my research and I know the best way to implement a high performance socket server is generally as follows: use async socket operations (specialized SocketAsyncEventArgs/Operations for best performance) and on the async callback, push the request to a processing queue that is tended by a pool of threads.

My questions for this processing model - to have the greatest performance:

1) when should the "End" operation of the socket be called (e.g. EndAccept or EndReceive)? should this be called on the callback thread (IOCP) before queuing the request? or called when the request gets taken out from the queue and gets processed (worker thread)?

2) this question depends on the answer to #1. when should the next "Begin" operation be called? should we call it before we call EndOperation even or should we call it immediately right after EndOperation (before queuing/processing request)?

3) can the processing queue simply be the .NET threadpool? what advantages/drawbacks would using the .NET threadpool vs. rolling out your own synchronized processing queue?

Any help is greatly appreciated.

shyneman
  • 305
  • 1
  • 3
  • 11
  • 1
    You didn't do your research well: SocketAsyncEventArgs and Begin/End are two different patterns. You don't call e.g. EndReceive when you called ReceiveAsync. – dtb Oct 14 '11 at 21:35
  • Regarding #1, `EndXXX` operations should be called inside the callback and outside as well (if the request completed synchrnoously). It's a bit tricky to get it right. See this [post](http://blogs.msdn.com/b/mjm/archive/2005/05/04/414793.aspx) for an explanation. If you want to avoid this and use a more elegant solution, I suggest you go with TPL and its `FromAsync`. – Ilian Oct 14 '11 at 22:50
  • 1
    @dtb you're correct the model is slightly different, but both patterns follow the overall concept of async processing via callbacks and my questions are still relevant to both – shyneman Oct 15 '11 at 00:37

1 Answers1

2

1) The EndReceive should be the first thing you do in the async callback. In fact, you can't do much of anything else in the callback until you've called EndReceive, because that's what gives you the data that was received. See the example at Socket.EndReceive.

Same thing goes for EndAccept, since it gives you the socket that you'll be communicating with.

2) You should call BeginAccept as soon as possible after the EndAccept. Otherwise you risk missing connection requests if your accept callback takes too long to process. Of course, if your accept callback is taking a long time to do anything, you're doing it wrong. Again, same thing goes for BeginReceive: call it as soon as you can after EndRead, to avoid losing data. Or, if your communications protocol is a request/response model, where the client expects a response before sending any more data, you could wait to call BeginRead until after you've sent the response.

3) The processing could be the .NET thread pool, although if you're going to use that you should look into using the Task Parallel Library. The advantage of using the TPL is that it's very much "fire and forget," or perhaps "fire and it'll call back when done." The disadvantage of using the TPL is that it's more difficult for your application to know what tasks are pending. If you create your own synchronized processing queue, you always know what jobs are pending and you have the potential of examining the queue, canceling jobs, etc. If you want to do that with the TPL, you end up creating a collection of tasks that you have to manage, since there's no way to get the the list of pending tasks.

But if you don't need to see the pending tasks, then TPL should work well.

The related questions here have some good information.

Community
  • 1
  • 1
Jim Mischel
  • 131,090
  • 20
  • 188
  • 351
  • Regarding #1, I would like to add though that the standard APM pattern is a bit more complicated than that. Because it is possible for the stack to overflow in naive implementations. For one, `EndXXX` methods should also be called outside the callback if the request completed synchronously. Here's a [blog](http://blogs.msdn.com/b/mjm/archive/2005/05/04/414793.aspx) about that. This is taken care of though if one uses TPL. – Ilian Oct 14 '11 at 22:46
  • For #1: if you queue the whole asyncresult without calling Endxxx, can't you free the IOCP thread earlier? For #2, I think we can agree that should call it asap so you can process the next connection in the accept buffer. For #3, I'll take a look into TPL... thanks to all that suggested it. – shyneman Oct 15 '11 at 00:39
  • @IlianPinzon So are you saying the Endxxx method should be called inside the processing worker thread NOT the socket IOCP thread? And thanks for suggesting TPL. – shyneman Oct 15 '11 at 00:52
  • @shyneman I'm not sure I understand you correctly. The `EndXXX` method should be called inside the callback if the request did not complete synchronously. If the request did complete synchronously, you have to call `EndXXX` where you called `BeginXXX`. The link I gave has the correct pattern. – Ilian Oct 15 '11 at 04:29
  • @IlianPinzon is it worth it to still put the request in a queue when the request processing is a CPU-bound? would that not just incur additional thread context switching with no additional output? – shyneman Oct 17 '11 at 09:19