Delay between receives when using asio::async_read

Question

I am using Boost/ASIO to write a C++ server and accompanying client app which talk over TCP/IP. I was seeing delays between consecutive receives which were causing lower-than-expected throughput between the server and the client. The code on either side looked roughly like

class MyStream
{
  ...

  void doStuff()
  {
    asio::async_read(socket, buffer, &bind(MyStream::readCallback, this, _1, _2)); // get some bytes
  }

  void readCallback(const error_code& err, size_t bytes_transferred)
  {
    processData(bytes_transferred);  // maybe write some data back

    asio::async_read(socket, buffer, &bind(MyStream::readCallback, this, _1, _2)); // get some additional bytes
  }

  ...
};

The messaging between the server and client was very slow, something like 20-30 messages back-and-forth per second. I was testing on the local machine, using very small messages.

Using the ASIO compiler flag -DASIO_ENABLE_HANDLER_TRACKING I observed delays between some of the receives which were consistently ~40ms.

...
@asio|1613871309.603585|>22|ec=asio.system:0,bytes_transferred=30
@asio|1613871309.603688|22^34|in 'ssl::stream<>::async_read_some' ([redacted]/asio/include/asio/ssl/detail/io.hpp:167)
@asio|1613871309.603688|22*34|socket@0x7fdebc013f80.async_receive
@asio|1613871309.603703|.34|non_blocking_recv,ec=asio.system:11,bytes_transferred=0
@asio|1613871309.603725|<22|
@asio|1613871309.643974|.34|non_blocking_recv,ec=asio.system:0,bytes_transferred=248
@asio|1613871309.644013|>34|ec=asio.system:0,bytes_transferred=248
...

What could be causing this 40ms delay?

Answer 1

The issue was that the messages are very small (a few bytes or less) and being written in multiple parts. This pattern, on a TCP connection with both Nagle's algorithm and delayed ACKs configured, results in delays between the separate sends, where the system is waiting for an ACK before sending more data, but there is an ACK delay (40ms by default on my system).

Edit 1:

As David Schwartz mentions in the comments the solution is to aggregate the outbound data.

Separately, one can do a sort-of-workaround by setting the TCP_NODELAY option, which disables Nagle's algorithm.

socket.set_option(asio::ip::tcp::no_delay(true));

There is also a TCP_QUICKACK flag, but from the docs:

This flag is not permanent, it only enables a switch to or from quickack mode. Subsequent operation of the TCP protocol will once again enter/leave quickack mode depending on internal protocol processing and factors such as delayed ack timeouts occurring and data transfer. This option should not be used in code intended to be portable.

Edit 2: For posterity-- I modified the code, once using the no_delay option, and once by changing the logic to do scatter-gather writes + quick acks after message-wise receives (the protocol I'm using does not have application-level acks). With only no_delay, throughput went from ~30 msg/s to ~300 msg/s. With only scatter-gather writes + quick acks, the throughput was about ~1000 msg/s. This is for my particular use case but I hope this gives some sense of what worked best for me.

Edit 3: After combining TCP_NODELAY with scatter-gather writes, I achieved the best throughput, something like 15% faster than scatter-gather with quick acks (which turns out to be a pretty slow system call)