std::ostream that invokes a callback for each line

Question

I am trying to write a custom std::ostream that invokes a function for every line written to it. That is, I would like the following code to work as documented in comments:

my_output_stream s([] (const std::string& line) { 
    WriteLineToSomeOutput(line); 
});

s << "Hello world"; // do not invoke anything. The line has not ended yet!
s << ", from Alex" << std::endl; // here we invoke WriteLineToSomeOutput("hello world, from Alex")
s << "What's up"; // do not invoke anything. The line has not ended yet.
s << ", doc?!\nHow u doing?\n"; // Now we have two lines. We invoke WriteLineToSomeOutput("What's up, doc?!) and WriteLineToSomeOutput("How u doing?")

Note that the data is not written anywhere and not stored anywhere. The only thing that I need the stream to store is the current line that is being aggregated, until we encounter an end of line.

I did not find any easy way of doing so, even when using the boost.Iostreams library. Can I avoid writing my own line tokenizer here by using some built-in facilities of STL and Boost?

Background

The my_output_stream class will be used to adapt between an external library and a logging library used in my application. The external library requires that I provide an std::ostream. And I want to log each line that is logged by the external library using my application's logging framework.

Answer 1

If I understand correctly, you want to unconditionally flush at end of line, and only at end of line. To do this, you must implement your own streambuf; it could be based on std::stringbuf, but if you're only concerned with output, and not worried about seeking, it's probably just as easy to do it yourself.

Something like the following should do the trick:

class LineBufferedOutput : public std::streambuf
{
    std::vector<char> myBuffer;
protected:
    int overflow( int ch ) override
    {
        myBuffer.push_back( ch );
        if ( ch == '\n' ) {
            //   whatever you have to do...
        }
        //  return traits::eof() for failure...
    }
};

I'm not sure what you mean by implementing your own tokenizer; there's no tokenization involved. You do have to look at each character, in order to compare it to '\n', but that's all.

And you ignore any explicit requests to sync().

Answer 2

I would probably start by implementing a streamable device and then wrapping that in a boost::iostream. Take a look at boost::iostreams and use this as a starter:

#include <iosfwd>                           // streamsize, seekdir
#include <boost/iostreams/categories.hpp>   // seekable_device_tag
#include <boost/iostreams/positioning.hpp>  // stream_offset

#include <boost/function.hpp>

class MyDevice
{

  public:
    typedef boost::function<void()> Callback; // or whatever the signature should be
    typedef char                                   char_type;
    typedef boost::iostreams::seekable_device_tag  category;

    explicit MyDevice(Callback &callback);

    std::streamsize read(char* s, std::streamsize n);
    std::streamsize write(const char* s, std::streamsize n);
    std::streampos seek(boost::iostreams::stream_offset off, std::ios_base::seekdir way);

  private:
    MyDevice();
    Callback myCallback;
};

That would be the basic declaration. You will need to define in your .cpp file how each function is implemented. One of these functions might be implemented as follows:

std::streampos MyDevice::write(const char* s, std::streamsize n)
{
    // process written data
    // file callback
    myCallback();
    // etc
}

Then to use from elsewhere, e.g. in your main function:

Callback callback; // some function
MyDevice device(callback);
boost::iostreams::stream<MyDevice> stream(device);
stream << data; // etc.

Answer 3

It appears you are just looking for line-buffering, and standard ostream already does that (unless you specifically request it not to using either std::unitbuf or std::flush manipulators).

Now, long lines could overflow the output buffer and trigger an "early" flush, but keep in mind, this can happen anyways: if output is to a file, the OS will apply the same kind of buffering strategies and it doesn't really matter in which chunks you flush long lines.

If output is to a socket, e.g. then you may send the buffer in one go (if you take good care to make sure the buffer is large enough) but the TCP/IP layer is free to break the stream up in packages depending on tuning and limits of your networking hardware.