Sending data to a program via stdin and ostream. (C++)

Question

I would like to send data from within my C++ program to an external pipeline, like so:

FILE* file = popen("my_prog -opt | other_prog", "w");
std::ostream fileStream = some_function(file);
fileStream << "some data";

I understand there is no simple, cross-platform way to do the second line, but is there any way to accomplish the same thing using something other than popen? I don't need to use popen, but I do need to use ostream. It would need to compile with clang and gcc at minimum, but preferably it would work with any compiler. I could also change how I handle the piping, but I don't have the source for my_prog or other_prog.

Answer 1

It is straight forward to create a stream buffer using a FILE* as underlying destination and create a corresponding std::ostream using such a stream buffer. It would roughly look something like this:

#include <stdio.h>
#include <streambuf>
#include <ostream>

class stdiobuf
    : public std::streambuf {
    enum { bufsize = 2048 };
    char buffer[bufsize];
    FILE* fp;
    int   (*close)(FILE*);
    int overflow(int c) {
        if (c != std::char_traits<char>::eof()) {
            *this->pptr() = std::char_traits<char>::to_char_type(c);
            this->pbump(1);
        }
        return this->sync()
            ? std::char_traits<char>::eof()
            : std::char_traits<char>::not_eof(c);
    }
    int sync() {
        std::streamsize size(this->pptr() - this->pbase());
        std::streamsize done(this->fp? fwrite(this->pbase(), 1, size, this->fp): 0);
        this->setp(this->pbase(), this->epptr());
        return size == done? 0: -1;
    }
public:
    stdiobuf(FILE* fp, int(*close)(FILE*) = fclose)
        : fp(fp)
        , close(close) {
        this->setp(this->buffer, this->buffer + (this->fp? bufsize - 1: 0));
    }
    ~stdiobuf() {
        this->sync();
        this->fp && this->close(this->fp);
    }
};
class opipestream
    : private virtual stdiobuf
    , public std::ostream {
public:
    opipestream(std::string const& pipe)
        : stdiobuf(popen(pipe.c_str(), "w"), pclose)
        , std::ios(static_cast<std::streambuf*>(this))
        , std::ostream(static_cast<std::streambuf*>(this)) {
    }
};

int main()
{
    opipestream out("/usr/bin/sed -e 's/^/xxxx /'");
    out << "Hello\n";
    out << "world\n";
}

The basic idea is that you can create a new stream by implementing a stream buffer. The implementation above should be fairly complete. The error handling when an incomplete buffer could be improved although the most likely case of an error is that the pipe was closed and there isn't really much what could be done.

Answer 2

My old answer did only work under Windows due to the missing std::filebuf ctor. As user4815162342 pointed out that an alternative might be using __gnu_cxx::stdio_filebuf<char>.

I patched something together that should now work with windows and linux, yet there is a chance that your platforms might not all work.

#ifdef __GNUC__
    #include <ext/stdio_sync_filebuf.h>    
    typedef __gnu_cxx::stdio_sync_filebuf<char> popen_filebuf;
#elif _MSC_VER
    #include<fstream>
    typedef std::filebuf popen_filebuf;
    FILE*(*popen)(const char*, const char*) = _popen;
#else
    static_assert(false, "popen_filebuf is not available for this platform");
#endif

int main()
{
    FILE* file = popen("my_prog -opt | other_prog", "w");

    popen_filebuf buffer(file);
    std::ostream fileStream(&buffer);

    fileStream << "some data";

    return 0;
}

Answer 3

If you know in advance which platforms you support, you can use platform-specific extensions to create an iostream out of a file descriptor. For example, as shown here, GNU libstdc++ provides __gnu_cxx::stdio_sync_filebuf which can be used to create a filebuf from a FILE *. A class that inherits from std::iostream and initializes it with the appropriate filebuf, tested with g++ 5.2 and clang++ 3.7 on Linux, can look like this:

#include <iostream>
#include <ext/stdio_sync_filebuf.h>

class StdioStream:
  private __gnu_cxx::stdio_sync_filebuf<char>, public std::iostream {
public:
  explicit StdioStream(FILE *fp):
    __gnu_cxx::stdio_sync_filebuf<char>(fp), std::iostream(this) { }
};

int main()
{
    FILE* file = popen("my_prog -opt | other_prog", "w");
    StdioStream fileStream(file);
    fileStream << "some data";
}

Note that one cannot just define a some_function that returns an std::ostream because the latter has a private copy constructor, and because the intermediate filebuf needs to be destructed along with the stream. The above example uses multiple inheritance to be able to initialize stdio_sync_filebuf (which would otherwise be a member) before the iostream base class - see base-from-member for details.

The referenced answer also contains equivalent code for MSVC++ iostreams.

Answer 4

An expansion on my comment to the original question.

You can save all the input into a file on disk, then feed that file as input to my_prog.

ofstream fileStream("input.txt");
fileStream << "some data";
// insert other input
fileStream.close();
FILE *file = popen("my_prog -opt <input.txt | other_prog", "w");

You could use something other than popen here, possibly a straight system call. Once you were all done, you'd want to delete the input.txt file.