How does Boost's async read/write/accept work in a multithreaded environment?

Question

I am currently writing a HTTP server. It blocks while waiting for socket-data (using boost::asio::ip::tcp::acceptor async_accept). Then when data is available creates a Session to handle the incoming request. The Session object calls Boost's http:async_read, a process_request method I wrote, then http::async_write.

We are trying to multithread the server. One idea would be to spawn X number of threads, all executing the start_accept function and listening on the same socket. Yet, I am not sure if this works using the Boost libraries (and can't figure it out from the docs).

TLDR; how does the Boost async read/write/accept functions work? Intuitively, I would think the OS routes incoming data to one of the threads, if each thread is blocking with async_accept. Are the async methods blocking across threads (and so thread safe)?

Here is the example where I am calling async_accept, then calling the Session object.

void Server::start_accept()
{
  // Start acceptor.
  m_acceptor.async_accept(m_socket, [&](beast::error_code error_code)
  {
    // After the accept operation completes
    if (!error_code) 
    {
      // Constructs a Session object then runs start()
      std::make_shared<Session>(std::move(m_socket), m_server_params)->start();
    }
    
    start_accept(); // Accept new request.
  });
}

I am envisioning some wrapper function that runs as follows:

void Server::start_multithreaded_server()
{
  // Spawn X threads
  create_thread(start_accept)
}

score 0 · Answer 1 · answered May 20 '22 at 03:32

Just run the io-service on multiple threads.

From there, you don't have to change a thing. The async_accept will be invoking the completion handler on any of the service threads.

Then since the only thing you do is initiate async operations via Session::start() the next async_accept will be posted immediately.

If for some reason you are doing work that blocks the service thread for non-trivial amount of time, e.g. in the Session constructor, that's a design smell (don't do that). But, still, if that were the case, you might move the socket first, and call start_accept even before you construct the Session.

Keep in mind that in a multi-threaded environment you will want to have all operations in the session on a (logical) strand. See Why do I need strand per connection when using boost::asio?

For inspiration you can look at the various server examples. Notably all of the server examples here https://www.boost.org/doc/libs/1_79_0/libs/beast/doc/html/beast/examples.html#beast.examples.servers are multi-threaded exactly as shown here, but show

the creation of threads

 // Run the I/O service on the requested number of threads
 std::vector<std::thread> v;
 v.reserve(threads - 1);
 for(auto i = threads - 1; i > 0; --i)
     v.emplace_back(
     [&ioc]
     {
         ioc.run();
     });
 ioc.run();

the organization of strands per session

How does Boost's async read/write/accept work in a multithreaded environment?

1 Answers1