asio::async_write and strand

Question

asio::async_write(m_socket, asio::buffer(buf, bytes),
                custom_alloc(m_strand.wrap(custom_alloc(_OnSend))));

Does this code guarantee that all asynchronous operation handlers(calls to async_write_some) inside async_write are called through strand? (or it's just for my_handler?)

score 9 · Accepted Answer · edited May 23 '17 at 10:27

9

With the following code:

asio::async_write(stream, ..., custom_alloc(m_strand.wrap(...)));

For this composed operation, all calls to stream.async_write_some() will be invoked within m_strand if all of the following conditions are true:

The initiating async_write(...) call is running within m_strand():

assert(m_strand.running_in_this_thread());
asio::async_write(stream, ..., custom_alloc(m_strand.wrap(...)));

The return type from custom_alloc is either:

the exact type returned from strand::wrap()

template <typename Handler> 
Handler custom_alloc(Handler) { ... }

a custom handler that appropriate chains invocations of asio_handler_invoke():

template <class Handler>
class custom_handler
{
public:
  custom_handler(Handler handler)
    : handler_(handler)
  {}

  template <class... Args>
  void operator()(Args&&... args)
  {
    handler_(std::forward<Args>(args)...);
  }

  template <typename Function>
  friend void asio_handler_invoke(
    Function intermediate_handler,
    custom_handler* my_handler)
  {
    // Support chaining custom strategies incase the wrapped handler
    // has a custom strategy of its own.
    using boost::asio::asio_handler_invoke;
    asio_handler_invoke(intermediate_handler, &my_handler->handler_);
  }

private:
  Handler handler_;
};

template <typename Handler>
custom_handler<Handler> custom_alloc(Handler handler)
{
  return {handler};
}

See this answer for more details on strands, and this answer for details on asio_handler_invoke.

edited May 23 '17 at 10:27

Community

1
1

answered Mar 15 '16 at 21:59

Tanner Sansbury

51,153
9
112
169

Please give source code or docs reference for "All calls to stream.async_write_some() will be invoked within m_strand". OP accepted this answer because he want this to be true ,but I dont think it is. – Galimov Albert Mar 16 '16 at 08:34
2

See my comments to your answer. It has to be this way or all kinds of things wouldn't work, starting with SSL. If the socket became readable and writable at the same time, two threads (one doing a composed operation for an async read and one doing one for an async write) might access the same SSL context at the same time and the program would crash. This is a core feature of ASIO and makes all kinds of things work. Go [here](http://www.boost.org/doc/libs/1_60_0/doc/html/boost_asio/overview/core/strands.html) and read the paragraph beginning "In the case of composed". – David Schwartz Mar 16 '16 at 09:32
@PSIAlt The [Strands: Use Threads Without Explicit Locking](http://www.boost.org/doc/libs/1_60_0/doc/html/boost_asio/overview/core/strands.html) page document this behavior. In this [answer](http://stackoverflow.com/a/32892581/1053968), I [demonstrate](http://coliru.stacked-crooked.com/a/034ade3310e37d53) intermediate handlers being invoked within the final completion handler's hook. – Tanner Sansbury Mar 16 '16 at 12:23
1

I love these tiny nit pick edits going on days after the post :) Very relatable. Making the answers even more valuable. Thank you again, Tanner! – sehe Mar 22 '16 at 08:04
@TannerSansbury, sorry, but I can't see why you put the first condition on all intermediate `async_write_some()` being invoked withing the strand, namely that the initiating `async_write()` call should already be under the strand. As per `asio_handler_invoke()` docs, doesn't it suffice that the completion handler goes through the strand? – Tarc Oct 14 '16 at 19:48
1

@Tarc The first call to `stream.async_write_some()` will occur within the `async_read()` initiating function. On the other hand, subsequent calls to `stream.async_write_some()` for the composed operation will occur within _intermediate handlers_ that run within the same context as the final completion handler. – Tanner Sansbury Oct 14 '16 at 22:45

Galimov Albert · Answer 2 · 2016-03-16T13:58:26.603

4

UPD: This answer is wrong=) For anyone interested you can check details in the tail of comments, thanks to Tanner Sansbury.

No. This guarantees your completion handler will be called with strand lock. This not includes calls to async_write_some.

Also boost::asio does not have something like "write queue" and you need to manage async_write calls in order to prevent writing mixed content.

edited Mar 16 '16 at 13:58

answered Mar 15 '16 at 16:32

Galimov Albert

7,269
1
24
50

what does this imply? It sounds like it would be unsafe to ever use `asio::async_write` in a multi-threading scenario. Why can that work? – sehe Mar 15 '16 at 19:21
@DavidSchwartz and it does not if you do concurrent async_write-s. If you look source code asio/impl/write.hpp you can see there is no strand for `async_write_some`. There is simply no need for this. Strands are for protection user code from concurrent execution, not for protecting socket. Dont forget to remove downvote after reading the source. – Galimov Albert Mar 16 '16 at 08:32
1

The issue is not concurrent writes, it's a read and write at the same time. The SSL connection only has one context and reading and writing at the same time would corrupt the SSL context. That's why you wrap in a strand, and the composed I/O functions inherit the protection through the `asio_handler_invoke` scheme. The downvote is appropriate -- your answer reflects a complete misunderstanding of how composed intermediate operations are dispatched and how the strand's magic invocation scheme protects them. – David Schwartz Mar 16 '16 at 09:26
Imagine this: I have an async read pending because I always do. I do an async write, but the buffer is full. At the same time, data is received on the socket and the write buffer empties a bit (an ACK can piggyback on data, so this is common). If the strand didn't protect the composed operations, you could wind up calling `SSL_read` and `SSL_write` at the same time, which would cause an immediate crash. Strand wrapping the completion handler and invoking on a strand automatically wraps the composed operations, protecting the SSL context from concurrent access. – David Schwartz Mar 16 '16 at 09:29
["IIn the case of composed asynchronous operations, such as async_read() or async_read_until(), if a completion handler goes through a strand, then all intermediate handlers should also go through the same strand. ... This is done by having hook functions for all intermediate handlers which forward the calls to the customisable hook associated with the final handler:"](http://www.boost.org/doc/libs/1_60_0/doc/html/boost_asio/overview/core/strands.html) Read the whole thing. – David Schwartz Mar 16 '16 at 09:31
@DavidSchwartz no, this is controlled by **asio::ssl::stream** code (check out `io_op` class in asio/ssl/detail/io.hpp). It encapsulates want_read/want_write, not the ASIO handlers themselves. Strands have nothing to do with this; `asio_handler_invoke` is used when operation is complete, not when it is in progress. – Galimov Albert Mar 16 '16 at 09:40
There's no locking in `io_op`. It's the strand that prevents a composed read and composed write from overlapping. At the reactor level, the SSL `io_op` is the thing that's invoked by `asio_handler_invoke` as it's the handler at that level. The strand wrapping of `asio_handler_invoke` just passes down the chain while the completion function is wrapped to form the composed operation. – David Schwartz Mar 16 '16 at 09:45
If this is not so, show me where you think a concurrent read and write are prevented from hitting the SSL engine. I have 100% tested this -- without the strand wrapping, you will get concurrent SSL_read/SSL_write invocations from the `io_op` and crash. – David Schwartz Mar 16 '16 at 09:46
@DavidSchwartz can you point me source snippet where "strand passes down to the chain" ? I see in asio/impl/write.hpp it does not. – Galimov Albert Mar 16 '16 at 09:49
1

I don't know what you're seeing in `write.hpp` that you think indicates that it's not. The docs say "SSL stream objects perform no locking of their own. Therefore, it is essential that all asynchronous SSL operations are performed in an implicit or explicit strand." You know a concurrent SSL_read and SSL_write on the same connection will cause a crash, right? – David Schwartz Mar 16 '16 at 09:53
3

Listen to the author of ASIO. Listen from [1:18:00 to 1:18:50](https://www.youtube.com/watch?v=D-lTwGJRx0o) as he explicitly states that the strand protects the internal composed operations and thus all the structures associated with the stream and the connection. This allows the whole connection and all its associated state to act as if it was single threaded. – David Schwartz Mar 16 '16 at 10:01
Also, a bit of common sense. If you were right, what purpose would `asio_handler_invoke` serve? The completion function would just be a function that posts whatever needs to be done to the strand. If you only had to invoke the completion handler and nothing else, there would be no reason to have both an invoker and a thing to be invoked. The reason you need a generic invoker is so that the wrapped completions that flow down the chain can be invoked by the reactor as the invoker also flows down the chain. – David Schwartz Mar 16 '16 at 10:04
@DavidSchwartz in asio/impl/write.hpp async_write works using async_write_some: `stream_.async_write_some(buffers_, *this);`. There is no strand or handler user has set as completion handler to `async_write`. Its different handler for `async_write_some` (a functor) and it does not uses user's strand. Also, i wont discuss anymore since it seems you highly rely on guesswork, not source code. – Galimov Albert Mar 16 '16 at 10:14
@DavidSchwartz btw thanks for the podcast link, anyway=) – Galimov Albert Mar 16 '16 at 10:14
6

@PSIAlt David seems to highly rely on knowledge of the design and the concepts behind it. Even if your analysis would amount to the discovery of a flaw in the implementation of `async_write` (unlikely), that would make it a bug, and the answer would still be incorrect. [The fallacy here is the assumption that you can better assess the working of a complicated library by looking at the implementation details than someone understanding the highlevel concepts. The odds of that are slim.] – sehe Mar 16 '16 at 10:33
@sehe Agreed. I had to look over all the details about a year or so ago, so I know it all works. Those details are not totally fresh in my mind. The basic idea is that any code other than the reactor can only be invoked two ways -- by high-level code calling it or by the reactor. High-level code must be sure only to call async functions when already on the strand. The reactor calls all non-reactor code with `asio_handler_invoke`, and the invoker flows down the call stack unmolested from your call to `strand_wrap` all the way to the reactor. – David Schwartz Mar 16 '16 at 10:45
4

For implementation details, in if the completion condition has not been met, then `write_op` will pass _itself_ as an _intermediate_ completion handler to an `async_write_some` operation. When `reactive_socket_send_op` completes, it will invoke `asio_handler_invoke(bound_handler, write_op)` that invokes `asio_handler_invoke(bound_handler, final_completion_handler)` that invokes `wrapped_handler.dispatcher.dispatch(bound_handler)` where `dispatcher` is a `strand` and `bound_handler` is a bound handler that invokes `write_op` with `ec` and `bytes_transferred`. – Tanner Sansbury Mar 16 '16 at 12:56
@TannerSansbury wow, thanks! Now this call chain clear to me. Updated the answ. – Galimov Albert Mar 16 '16 at 13:56
3

Since the edit I've upvoted this answer, because it's no longer misleading, and I think the comment exchange is good to keep! Thanks guys for sticking to the facts with patience. /cc @DavidSchwartz – sehe Mar 17 '16 at 07:46

asio::async_write and strand

2 Answers2

Linked