Use std::istream and std::ostream between threads without system calls (pipe)

Question

I am looking for a simple way to create a stream of communication between threads using <iostream> semantics. I was looking for something like the following:

#include <iostream>
#include <thread>

void thread1(std::istream from_main, std::ostream to_main) {
    std::string s;
    from_main >> s;
    to_main << "Received:" << s << std::endl;
}
int main() {
   std::istream from_thread;
   std::ostream to_thread;
   std::thread t(thread1, to_thread, from_thread);
   to_thread << "Hi-Thread\n";
   std::string s;
   from_thread >> s; // Received:Hi-Thread
   t.join();
}

Is there a simple way of achieving this without using pipe, creating file descriptors and filling up the code with system calls?

I'm curious why you'd need pipes, fds and system calls to do this? Everything is in the same process, so you could have a queue of messages and a lock, right? What is the use case here? — ggorlen, Oct 05 '19 at 21:51
@ggorlen True, but then I miss the whole `std::stream` semantics, which are quite nice. — Kostas, Oct 05 '19 at 21:52
@ggorlen I certainly could, but this introduces possibilities for all kinds of mistakes on my part. I would be quite surprised if there isn't a more standard solution to this. — Kostas, Oct 05 '19 at 21:55
Sure, this seems like a complex abstraction, but again it's sort of unclear what the point of all of this is. It's worth [mentioning what you're really trying to achieve here](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem). Said differently, if there's no standard abstraction for doing something like this, it tends to indicate that it's not a particularly good solution to whatever problem you're trying to solve. — ggorlen, Oct 05 '19 at 21:57
@ggorlen I came across this when trying to create a packet delegation system, where each thread sends and receives packets for it's specific purpose, while main handles the delegation of these packets (filtering by header). There are different ways of doing this, but this seems like the most obvious and intuitive, and hopefully avoids getting the operating system too involved (apart from the I/O blocking part). — Kostas, Oct 05 '19 at 22:03
Seems like it would be reasonably straight-forward to define a subclass of `std::streambuf` that has the necessary behavior. The you could just constuct an istream and ostream that refer to it. — Chris Dodd, Oct 05 '19 at 22:12
What's wrong with using pipes? I'm pretty sure some C++ standard library implementations provide ways to open an iostream using a file descriptor along the lines of `fdopen()` for stdio streams, or there's always Boost.Iostreams which can do the same without depending on a specific implementation. — Shawn, Oct 05 '19 at 22:31
The whole `std::stream` semantics are not as nice as you make them out to be. Just try sending `"Hi Thread"` instead of `"Hi-Thread"` — Igor Tandetnik, Oct 06 '19 at 01:28

score 2 · Accepted Answer · answered Oct 06 '19 at 02:01

It looks like what you need is a single-producer, single-consumer (SPSC) queue, possibly lock-free. I'd start with that, and if you feel a strong need to build up syntactic sugar to make operator<< mean spsc_queue::push_back, add that on later. Don't start from the perspective of the C++ operator << syntax and then think "Oh that means std::ostream" which then leads to "Let's make a custom streambuf."

Keep it simple. SPSC queue.

Use std::istream and std::ostream between threads without system calls (pipe)

1 Answers1