3

I just started working with the Boost ASIO library, version 1.52.0. I am using TCP/SSL encryption with async sockets. From other questions asked here about ASIO, it seems that ASIO does not support receiving a variable length message and then passing the data for that message to a handler.

I'm guessing that ASIO puts the data into a cyclical buffer and loses all track of each separate message. If I have missed something and ASIO does provide a way to pass individual messages, then please advise as to how.

My question is that assuming I can't somehow obtain just the bytes associated with an individual message, can I use transfer_exactly in async_read to obtain just the first 4 bytes, which our protocol always places the length of the message. Then call either read or async_read (if read won't work with async sockets) to read in the rest of the message? Will this work? Any better ways to do it?

Bob Bryan
  • 3,687
  • 1
  • 32
  • 45
  • 3
    This is not an issue with ASIO, but with TCP itself. TCP is a streaming protocol, so you aren't guaranteed to receive a full "message" (in the application sense) with a single call to `recv()`, be that synchronous or not. However, you should look into the free functions `boost::asio::async_read()` which can provide guarantees to return when the buffer is fully populated, handled by multiple underlying calls to `socket.async_recv()`. – Chad Jan 18 '13 at 20:41
  • Also, see this related question: http://stackoverflow.com/questions/4933610/boostasio-async-read-guarantee-all-bytes-are-read – Chad Jan 18 '13 at 20:42
  • I have tried using async_read from the examples. I am sending one message out and expecting 5 back at startup. So, it seems to place all these messages in one buffer. I did try using transfer_at_least(4) in async_read. It then sent 48 bytes, which is the total amount for the 1st message. But, further calls to async_read resulted in it filling the entire buffer with data from multiple messages. I will try my idea of using transfer_exactly to just get the message length and then do another read to get the rest of it. – Bob Bryan Jan 18 '13 at 20:55
  • I have used other libraries with TCP in the past. I seem to remember that they were able to keep track of individual messages and pass each of them to a handler intact. Maybe I will submit a feature request for this since it seems like a very logical way to do things. – Bob Bryan Jan 18 '13 at 20:58
  • 2
    @BobBryan very likely these libraries built their own protocol on top of TCP, like a length-prefixed protocol, RAW TCP is not message aware, it's not a part of the protocol and no matter how good your implementation is, you can't megically send raw messages over tcp without some metadata. – KillianDS Jan 18 '13 at 21:25
  • @KillianDS, yes you are probably right. It has been a while since I last worked with sockets. – Bob Bryan Jan 18 '13 at 23:16
  • I just tried my idea and it seems to work. But, I am thinking that perhaps @DanS has a better approach. The problem with my idea is that it will wait for a certain number of bytes to accumulate before it will transfer the data from the socket buffer to my application buffer. For my specific application, the average number of bytes per message is not much more than 120, and most are less than half that. If the message size were greater, then it could run out of memory or lose data while waiting for the rest of the message to come in. – Bob Bryan Jan 18 '13 at 23:43

1 Answers1

4

Typically I like to take the data I receive in an async_read and put it in a boost::circular_buffer and then let my message parser layer decide when a message is complete and pull the data out. http://www.boost.org/doc/libs/1_52_0/libs/circular_buffer/doc/circular_buffer.html

Partial code snippets below

boost::circular_buffer TCPSessionThread::m_CircularBuffer(BUFFSIZE);

void TCPSessionThread::handle_read(const boost::system::error_code& e, std::size_t bytes_transferred)
{
    // ignore aborts - they are a result of our actions like stopping
    if (e == boost::asio::error::operation_aborted)
        return;
    if (e == boost::asio::error::eof)
    {
        m_SerialPort.close();
        m_IoService.stop();
        return;
    }
    // if there is not room in the circular buffer to hold the new data then warn of overflow error
    if (m_CircularBuffer.reserve() < bytes)
    {
        ERROR_OCCURRED("Buffer Overflow");
        m_CircularBuffer.clear();
    }
    // now place the new data in the circular buffer (overwrite old data if needed)
    // note: that if data copying is too expensive you could read directly into
    // the circular buffer with a little extra effor
    m_CircularBuffer.insert(m_CircularBuffer.end(), pData, pData + bytes);
    boost::shared_ptr<MessageParser> pParser = m_pParser.lock(); // lock the weak pointer
    if ((pParser) && (bytes_transferred)) 
        pParser->HandleInboundPacket(m_CircularBuffer); // takes a reference so that the parser can consume data from the circ buf
    // start the next read
    m_Socket.async_read_some(boost::asio::buffer(*m_pBuffer), boost::bind(&TCPSessionThread::handle_read, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));
}
DanS
  • 1,221
  • 1
  • 9
  • 4
  • Not quite as ignored as they look, since there are isochronous messages in this particular protocol and error handling is pretty much tear down and recreate the connection for just about any case where a message is not received within the timeout period. – DanS Jan 18 '13 at 21:22
  • Yes, but in this case the locally created instance of `ignored_ec` is not used. – Chad Jan 18 '13 at 21:23
  • Ah, I see what you mean. That was a dangling reference from code that I cut out as irrelevant to the example where it was, in fact, created, filled in, and logged but otherwise ignored as opposed to what I posted where it is just created ignored and destroyed for the fun of it :) Post edited .. thanks. – DanS Jan 18 '13 at 21:27
  • @DanS Do you have an example for reading directly into a `boost::circular_buffer` with `boost::asio`? – Robert Hegner Nov 08 '13 at 10:11
  • @DanS What does `bytes`,`pData` and `m_pBuffer` refer to in this example? – Don Scott Nov 16 '14 at 08:56
  • Using circular buffer is needed, there isn't any better approach for this? There is no other way to know when a TCP message ended? I mean something like UART_IDLE in embedded devices, or similar. – kurta999 Sep 30 '22 at 12:52