0

I am encountering an issue with the boost beast websocket stream. The issue occurs intermittently when I try to write to a stream in which the remote endpoint has stopped responding (specifically due to the remote being physically disconnected from the network).

When this issue occurs, a synchronous stream.write() call eventually hangs for a very long time (minutes) until the socket ultimately closes. I understand that this behavior is likely because my program continues to write to the stream without acks from the remote until the send buffer is full. I am wondering if there is a way to either apply a timeout to the write call, or if there is an interface more along the lines of stream.try_write() where I can raise error handling to the user-level.

I do realize that one option is to use the async_write interface. However, I am concerned that this will negatively impact my send performance by deferring the socket write operation until the next iteration of io_context.

Below is a stack-trace of the thread when the call has hung.

#0  0x00007f468cf33624 in poll () from /lib64/libc.so.6
#1  0x000000000043e5a7 in boost::asio::detail::socket_ops::poll_write (ec=..., msec=-1, state=0 '\000', s=16)
    at /usr/include/boost/asio/detail/impl/socket_ops.ipp:1898
#2  boost::asio::detail::socket_ops::sync_send (ec=..., all_empty=<optimized out>, flags=0, count=<optimized out>, bufs=0x7fff43c17e20, 
    state=<optimized out>, s=<optimized out>) at /usr/include/boost/asio/detail/impl/socket_ops.ipp:1224
#3  boost::asio::detail::reactive_socket_service_base::send<boost::asio::detail::prepared_buffers<boost::asio::const_buffer, 64ul> > (impl=..., 
    buffers=..., ec=..., this=<optimized out>, flags=0) at /usr/include/boost/asio/detail/reactive_socket_service_base.hpp:245
#4  0x0000000000481c71 in boost::asio::basic_stream_socket<boost::asio::ip::tcp>::write_some<boost::asio::detail::prepared_buffers<boost::asio::const_buffer, 64ul> > (ec=..., buffers=..., this=0x108ad50) at /usr/include/boost/asio/buffer.hpp:941
#5  boost::asio::detail::write_buffer_sequence<boost::asio::basic_stream_socket<boost::asio::ip::tcp>, boost::beast::buffers_cat_view<boost::asio::mutable_buffer, boost::beast::buffers_prefix_view<boost::beast::buffers_suffix<boost::beast::basic_multi_buffer<std::allocator<char> >::const_buffers_type> > >, boost::beast::buffers_cat_view<boost::asio::mutable_buffer, boost::beast::buffers_prefix_view<boost::beast::buffers_suffix<boost::beast::basic_multi_buffer<std::allocator<char> >::const_buffers_type> > >::const_iterator, boost::asio::detail::transfer_all_t> (completion_condition=..., ec=..., buffers=..., s=...)
    at /usr/include/boost/asio/impl/write.hpp:53
#6  boost::asio::write<boost::asio::basic_stream_socket<boost::asio::ip::tcp>, boost::beast::buffers_cat_view<boost::asio::mutable_buffer, boost::beast::buffers_prefix_view<boost::beast::buffers_suffix<boost::beast::basic_multi_buffer<std::allocator<char> >::const_buffers_type> > >, boost::asio::detail::transfer_all_t> (ec=..., buffers=..., s=..., completion_condition=...) at /usr/include/boost/asio/impl/write.hpp:69
#7  boost::asio::write<boost::asio::basic_stream_socket<boost::asio::ip::tcp>, boost::beast::buffers_cat_view<boost::asio::mutable_buffer, boost::beast::buffers_prefix_view<boost::beast::buffers_suffix<boost::beast::basic_multi_buffer<std::allocator<char> >::const_buffers_type> > > > (ec=..., buffers=..., s=...)
    at /usr/include/boost/asio/impl/write.hpp:92
#8  boost::beast::websocket::stream<boost::asio::basic_stream_socket<boost::asio::ip::tcp> >::write_some<boost::beast::basic_multi_buffer<std::allocator<char> >::const_buffers_type> (this=this@entry=0x108ad50, fin=fin@entry=true, buffers=..., ec=...) at /usr/include/boost/beast/websocket/impl/write.ipp:625
#9  0x000000000042c5e1 in boost::beast::websocket::stream<boost::asio::basic_stream_socket<boost::asio::ip::tcp> >::write<boost::beast::basic_multi_buffer<std::allocator<char> >::const_buffers_type> (ec=..., buffers=..., this=0x108ad50)
miles.sherman
  • 171
  • 3
  • 11
  • If you just `run()` the `io_context`, then obviously theere will not be more of a delay. Also, doing thing asynchronous is a much better scaling proposition, so unless you are intending to "zero"-latency IO to a single endpoint it will likely perform better – sehe Sep 24 '18 at 00:02
  • My goal is latency, my bandwidth requirement is low. I am within the context of io_context::run(), but there are many other callback which can be invoked between the proposed async_write() call and the next iteration of the service. – miles.sherman Sep 24 '18 at 01:06
  • Then just don't run all those "many" other things on the same service. As far as I've been able to observe, some services even do immediately perform their action (at least UDP async_send_to does appear to have effect immediately, even without running the io_context) – sehe Sep 24 '18 at 01:44
  • There are other implications of using async_send, such as having to manage the memory of the send buffer instead of calling write() on a stack buffer. If synchronous write() is something that boost wants to support, there should be a solution for this issue because it renders the interface unusable. – miles.sherman Sep 24 '18 at 12:22
  • was able to resolve this issue, see answer i just posted. – miles.sherman Sep 24 '18 at 13:08

2 Answers2

1

Beast websockets does not support non-blocking mode. The implementation of websocket::stream will produce undefined behavior in some cases if you set this mode on a socket used with a websocket stream. The lack of timeouts is a general problem with synchronous code. You really have no choice other than to use asynchronous operations. You said you want to send buffers directly from the stack, this is easily accomplished in asynchronous contexts by using coroutines (see boost::asio::spawn and boost::asio::yield_context).

There are plenty of techniques which allow asynchronous I/O to perform the same or better than synchronous I/O, including for the case of low desired latency.

Here is advice directly from the author of Boost.Asio on achieving ultra-low latency: https://groups.google.com/a/isocpp.org/d/msg/sg14/FoLFHXqZSck/i4rdO-O3BQAJ

Vinnie Falco
  • 5,173
  • 28
  • 43
  • thanks for the follow-up vinnie, this helped me resolve the issue. – miles.sherman Oct 03 '18 at 21:27
  • 1
    Out of curiously, can you elaborate on why non-blocking sockets are not supported? – miles.sherman Oct 03 '18 at 23:31
  • 1
    Yes. Beast websocket operations can involve both reads and writes. For example, when performing a read, the implementation will automatically respond to pings and close frames. This means that when you perform a read, the implementation may need to also perform writes for you. You can still do writes yourself, but these user initiated writes are scheduled with the other writes performed by beast. For these reasons it is not practical to support the non-blocking mode. – Vinnie Falco Oct 04 '18 at 14:31
  • @VinnieFalco the google groups link you shared doesn't seem to work for me; do you know if the advice is preserved/available anywhere else please? – Steve Lorimer May 18 '22 at 08:48
  • @VinnieFalco never mind - I found that you shared it here: https://stackoverflow.com/a/54847003/955273 Thanks! :) – Steve Lorimer May 18 '22 at 08:56
-1

I was able to solve this issue by putting the underlying stream socket into non-blocking mode:

socket.non_blocking(true);

In this mode, once the send buffer becomes full, the write() call will return boost::system::error_code::try_again (a.k.a. posix EAGAIN) immediately.

miles.sherman
  • 171
  • 3
  • 11