Yield boost::asio::coroutine until task complete

Question

I'm building a simple C++ server where I receive an image via a POST request, send it to a GPU for processing, and once I have the results from the GPU, send back a response.

To be able to handle many simultaneous connections (and to learn something new), I'm using Boost::asio, following the server4 example (link) that uses boost::asio::coroutine. The issue I'm running into is how to send data from the coroutine to the GPU while not blocking the coroutine.

The GPU is most efficient when it can process a batch of requests together. But even processing one-request-at-a-time (cannot be context-switched as a CPU and memory IO is the bottleneck), it must process a complete request before starting on the next one. This means I need to queue the requests from the coroutines, and signal the coroutine somehow when the GPU processing is complete.

Been looking through the boost::asio reference but nothing is jumping out at me. In general, how are boost asio coroutines used where a response cannot be generated immediately (e.g. may take ~500ms). How is this typically done?

Answer 1

Set up the part of the program that interacts with the GPU to run on it's own thread. When you recieve a request that requires the GPU, box up the image data, along with the socket object and other required parameters, and push it into a (probably mutex-locked) queue. The gpu thread can then pull from the queue, perform its work, then spawn a new task on the io_service to finish the transaction with the client, using the information stored in the structure.

The gpu thread can be set to sleep until work becomes available using a condition variable and having the network code invoke notify_one() after it has pushed the work request.

Answer 2

Another trick that works exactly as you describe is to set a deadline timer with an infinite deadline, which you await in the coro (yielding).

Then from the GPU task you cancel the time when the work is complete, resuming the coro.

See for example

• Why does Boost.Asio not support an Event-based interface (for example, something along the lines of condition variable signaling)?
• Can a boost::asio::yield_context be used as a deadline_timer handler when doing cancel?