Create std::stringbuf based on std::vector

Question

I am doing in-memory image conversions between two frameworks (OpenSceneGraph and wxWidgets). Not wanting to care about the underlying classes (osg::Image and wxImage), I use the stream oriented I/O features both APIs provide like so:

1) Create an std::stringstream

2) Write to the stream using OSG's writers

3) Read from the stream using wxWigdets readers

This works fairly well. Until now I've been using direct access to the stream buffer, but my attention has been recently caught by the "non-contiguous underlying buffer" problem of the std::stringstream. I had been using a kludge to get a const char* ptr to the buffer - but it worked (tested on Windows, Linux and OSX, using MSVC 9 and GCC 4.x), so I never fixed it.

Now I understand that this code is a time bomb and I want to get rid of it. This problem has been brought up several times on SO (here for instance), but I could not find an answer that could really help me do the simplest thing that could possibly work.

I think the most reasonable thing to do is to create my own streambuf using a vector behind the scenes - this would guarantee that the buffer is contiguous. I am aware that this would not a generic solution, but given my constraints:

1) the required size is not infinite and actually quite predictable

2) my stream really needs to be an std::iostream (I can't use a raw char array) because of the APIs

anybody knows how I can either a custom stringbuf using a vector of chars ? Please do not answer "use std::stringstream::str()", since I know we can, but I'm precisely looking for something else (even though you'd say that copying 2-3 MB is so fast that I wouldn't even notice the difference, let's consider I am still interested in custom stringbufs just for the beauty of the exercise).

score 3 · Accepted Answer · answered Mar 26 '12 at 08:52

3

If you can use just an istream or an ostream (rather than bidirectional), and don't need seeking, it's pretty simple (about 10 lines of code) to create your own streambuf using std::vector<char>. But unless the strings are very, very large, why bother? The C++11 standard guarantees that std::string is contiguous; that a char* obtained by &myString[0] can be used to as a C style array.` And the reason C++11 added this guarantee was in recognition of existing practice; there simply weren't any implementations where this wasn't the case (and now that it's required, there won't be any implementations in the future where this isn't the case).

answered Mar 26 '12 at 08:52

James Kanze

150,581
18
184
329

I'm not sure I understand you answers. About using istream/ostream only: one plugin needs to see an ostream, while the other an istream. Maybe I can build one of each, using the same underlying vector ? About using std::string: OK, but how do you create streams out of a string ? – Mar 26 '12 at 08:57
@If one plugin needs an ostream, the other an istream, then you may have to make something bidirectional. Or if they don't need them simultaneously, then create first an ostream, then the istream, using the same buffer. As for streams out of a string: `std::[io]stringstream` is already there. – James Kanze Mar 26 '12 at 09:47
Do you mean that the buffer of the std::stringstream, according to C++11, is guaranteed to be contiguous ? And nowhere could I read that std::stringstreams used std::strings internally - any pointer ? – Mar 26 '12 at 11:41
The buffers of all `streambuf` are guaranteed to be contiguous. Otherwise, the interface couldn't work. But that's not the issue; you don't care what the buffer looks like, since unless you are deriving from `streambuf`, you can't see it. – James Kanze Mar 26 '12 at 12:26

score 1 · Answer 2 · answered Mar 26 '12 at 09:22

boost::iostreams have a few ready made sinks for this. There's array_sink if you have some sort of upper limit and can allocate the chunk upfront, such a sink won't grow dynamically but on the other hand that can be a positive as well. There's also back_inserter_device, which is more generic and works straight up with std::vector for example. An example using back_inserter_device:

#include <string>
#include <iostream>
#include "boost/iostreams/stream_buffer.hpp"
#include "boost/iostreams/device/back_inserter.hpp"
int main() 
{
    std::string destination;
    destination.reserve( 1024 ); 
    boost::iostreams::stream_buffer< boost::iostreams::back_insert_device< std::string > > outBuff( ( destination ) );
    std::streambuf* cur = std::cout.rdbuf( &outBuff );
    std::cout << "Hello!" << std::endl;
    // If we used array_sink we'd need to use tellp here to retrieve how much we've actually written, and don't forgot to flush if you don't end with an endl!
    std::cout.rdbuf( cur );
    std::cout << destination;
}

Thanks. I'm not using boost in this project but will keep your solution under my pillow for future ones. — , Mar 26 '12 at 11:38

Create std::stringbuf based on std::vector

2 Answers2