Socket "Flush" by temporarily enabling NoDelay

Question

Background

I have an implementation of an HTTP server in C#. Using ab I discovered a weird performance issue. Each request took 5 ms with Keep-Alive Off but 40 ms with Keep-Alive on!

The testpage is generated into a single byte[] which get sent as reply using a single socket.Send call.

The cause is as far as I can tell Nagle's algorithm used in the TCP stack.

TCP Flush?

So far I am using the NoDelay property in the end of every HTTP request served.

socket.NoDelay = true;
socket.NoDelay = false;

Which does solve the problem for now. But I have no documentation to backup my discovery.

This was tested on a linux/mono system.

Is there a standard way of flushing the TCP connection?

Related

This answer is addressing the same issue. The difference here is that I am looking to only temporarily disabling the algorithm.

Answer 1

I tested this with Wireshark. Unfortunately,

socket.NoDelay = true;
socket.NoDelay = false;

has no effect. Similarly,

socket.NoDelay = true;
socket.Send(new byte[0]);
socket.NoDelay = false;

also has no effect. From observed behaviour, it appears that the NoDelay property only affects the next call to Send with a non-empty buffer. In other words, you have to send some actual data before NoDelay will have any effect.

Therefore, I conclude that there is no way to explicitly flush the socket if you don’t want to send any extra data.

However, since you are writing an HTTP server, you may be able to use a few tricks:

• For requests that are served using Transfer-Encoding: chunked, you can send the end-of-stream marker (the "\r\n0\r\n\r\n") with NoDelay = true.
• If you are serving a file from the local filesystem, you will know when the file ends, so you could set NoDelay = true just before sending the last chunk.
• For requests that are served using Content-Encoding: gzip, you can set NoDelay = true just before closing the gzip stream; the gzip stream will send some last bits before actually finishing and closing.

I’m certainly going to add the above to my HTTP server now :)

Answer 2

If you know the the length of the data you are sending,Setting SendBufferSize will make the socket send the data at once, Following is a working code example:

byte[] bytesToSend = ASCIIEncoding.ASCII.GetBytes(message);
socket.SendBufferSize = bytesToSend.Length;
socket.Send(bytesToSend);

Answer 3

Having written a quite popular web server myself I don't think that Nagle algortihm is your real problem.

How do you build your responses and how do you send them?

Answer 4

There is no such thing as flushing in TCP. TCP is a stream based protocol which groups/ungroups/splits/joins your data. By disabling nagle it will only do that less frequent.

Do not disable nagle.

The Nagle TCP/IP algorithm was designed to avoid problems with small packets, called tinygrams, on slow networks. The algorithm says that a TCP/IP connection can have only one outstanding small segment that has not yet been acknowledged. The definition of "small" varies but usually it is defined as "less than the segment size" which on ethernet is about 1500 bytes.

Take a look at here: Disabling TCP/IP Nagle Algorithm Improves Speed on Slow Nets