Creating a std::iostream adapter

Question

I'd like to create an iostream adapter class which lets me modify the data written to or read from a stream on-the-fly. The adapter itself should be a iostream to allow true transparency towards third-party code.

Example for a StreamEncoder class derived from std::ostream:

// External algorithm, creates large amounts of log data
int foo(int bar, std::ostream& logOutput);

int main()
{
    // The target file
    std::ofstream file("logfile.lzma");
    // A StreamEncoder compressing the output via LZMA
    StreamEncoder lzmaEncoder(file, &encodeLzma);
    // A StreamEncoder converting the UTF-8 log data to UTF-16
    StreamEncoder utf16Encoder(lzmaEncoder, &utf8ToUtf16);

    // Call foo(), but write the log data to an LZMA-compressed UTF-16 file
    cout << foo(42, utf16Encoder);
}

As far as I know, I need to create a new basic_streambuf derivate and embed it in a basic_ostream subclass, but that seems to be pretty complex.

Is there any easier way to accomplish this?

Have you seen the [boost iostreams](http://www.boost.org/libs/iostreams) library? That already has compressing filters (no lzma AFAIK though) and mechanics for making writing custom filters much simpler. — Flexo, Jul 15 '12 at 15:36
It seems you should study the concept of stream manipulators. Search SO for it, there's an example here http://stackoverflow.com/questions/799599/c-custom-stream-manipulator-that-changes-next-item-on-stream or here http://stackoverflow.com/questions/535444/custom-manipulator-for-c-iostream — Johan Lundberg, Jul 15 '12 at 15:20
The right way to do this really does invole writing a custom `streambuf`. This really isn't a complicated task; really the only difficulty is finding references that describe what needs to be done. — , Jul 15 '12 at 18:45

score 4 · Accepted Answer · answered Jul 15 '12 at 20:19

Oddly enough, at least as things are really intended to work, none of this should directly involve iostreams and/or streambufs at all.

I would think of an iostream as a match-maker class. An iostream has a streambuf which provides a buffered interface to some sort of external source/sink of data. It also has a locale, which handles all the formatting. The iostream is little more than the playground supervisor that keeps those two playing together nicely (so to speak). Since you're dealing with data formatting, all of this is (or should be) handled in the locale.

A locale isn't monolithic though -- it's composed of a number of facets, each devoted to one particular part of data formatting. In this case, the part you probably care about is the codecvt facet, which is used (almost exclusively) to translate between the external and internal representations of data being read from/written to iostreams.

For better or worse, however, a locale can only contain one codecvt facet at a time, not a chain of them like you're contemplating. As such, what you really need/want is a wrapper class that provides a codecvt as its external interface, but allows you to chain some arbitrary set of transforms to be done to the data during I/O.

For the utf-to-utf conversion, Boost.locale provides a utf_to_utf function, and codecvt wrapper code, so doing this part of the conversion is simple and straightforward.

Lest anybody suggest that such things be done with ICU, I'll add that Boost.Locale is pretty much a wrapper around ICU, so this is more or less the same answer, but in a form that's much more friendly to C++ (whereas ICU by itself is rather Java-like, and all but overtly hostile to C++).

The other side of things is that writing a codecvt facet adds a great deal of complexity to a fairly simple task. A filtering streambuf (for one example) is generally a lot simpler to write. It's still not as easy as you'd like, but not nearly as bad as a codecvt facet. As @Flexo already mentioned, the Boost iostreams library already includes a filtering streambuf that does zip compression. Doing roughly the same with lzma (or lzh, arithmetic, etc. compression) is relatively easy, at least assuming you have compression functions that are easy to use (you basically just supply them with a buffer of input, and they supply a buffer of results).

Creating a std::iostream adapter

1 Answers1