33

I have some data in a buffer pointed to by a const char* pointer. The data is just an ASCII string. I know its size. I would like to be able to read it in the same way data is read from streams. I'm looking for a solution that would allow me to write code like this:

// for example, data points to a string "42 3.14 blah"
MemoryStreamWrapper in(data, data_size);
int x;
float y;
std::string w;
in >> x >> y >> w;

Important condition: the data must not be copied or altered in any way (otherwise I'd just use a string stream. To my best knowledge, it isn't possible to create a string stream from a const char pointer without copying the data.)

2 Answers2

68

The way to do this is to create a suitable stream buffer. This can, e.g., be done like this:

#include <streambuf>
#include <istream>

struct membuf: std::streambuf {
    membuf(char const* base, size_t size) {
        char* p(const_cast<char*>(base));
        this->setg(p, p, p + size);
    }
};
struct imemstream: virtual membuf, std::istream {
    imemstream(char const* base, size_t size)
        : membuf(base, size)
        , std::istream(static_cast<std::streambuf*>(this)) {
    }
};

The only somewhat awkward thing is the const_cast<char*>() in the stream buffer: the stream buffer won't change the data but the interface still requires char* to be used, mainly to make it easier to change the buffer in "normal" stream buffers. With this, you can use imemstream as a normal input stream:

imemstream in(data, size);
in >> value;
Dietmar Kühl
  • 150,225
  • 13
  • 225
  • 380
  • What does this get you that `istringstream`'s `pubsetbuf` doesn't? – ildjarn Oct 24 '12 at 23:08
  • Can't you just create a normal `std::istringstream` and through `.rdbuf()->pubsetbuf(data, size)` do the same? – Xeo Oct 24 '12 at 23:09
  • That was easier than I though! I guess I thought it would be complicated because I've done the same for socket streams, in which case it does require hundreds of lines... – user1610015 Oct 24 '12 at 23:09
  • 3
    @ildjarn: it has defined behavior on all platforms rather than different behavior on different platforms: `std::stringbuf::setbuf()` is implementation defined according to 27. 8.2.4 [stringbuf.virtuals] paragraph 15. – Dietmar Kühl Oct 24 '12 at 23:10
  • @user1610015: depends on what you want to do. I'd think that a socket stream doing blocking reads and writes take maybe 50 lines of codes. Playing with asynchronous reads and writes is a different things, though, and using a stream buffer probably doesn't quite work. – Dietmar Kühl Oct 24 '12 at 23:12
  • @MooingDuck: Thanks! ;) Primarily it is, yet, untested, though: I'm sure I have some typos somewhere... (testing is next) - OK, there was just a small typo (`std::ostream` vs. `std::istream` in the initializer list). – Dietmar Kühl Oct 24 '12 at 23:14
  • Ah, I found the source of my confusion: http://stackoverflow.com/a/5154222/636019 – ildjarn Oct 24 '12 at 23:18
  • 1
    @ildjarn: I guess, stackoverflow is a bit like Wikipedia in that respect: there is lot of really useful information but not everything is always entirely correct! Maybe I should put my warning about the behavior being implementation defined there as well... - oh, you already did! ;) – Dietmar Kühl Oct 24 '12 at 23:21
  • @Dietmar : Well, I'm glad you cleared things up (and that I've never relied on that erroneous trick in real code). :-] – ildjarn Oct 24 '12 at 23:23
  • 2
    Put this in C++15 while you're at it. – Mooing Duck Oct 24 '12 at 23:31
  • @DietmarKühl Well, it may also have to do with the fact that your coding style is pretty compact :) (using `struct` instead of `class`, omitting access modifiers, etc.) – user1610015 Oct 24 '12 at 23:32
  • 2
    @MooingDuck: last week I learned that we are targetting C++ 2014 at the moment! That is, complete proposals, ready to go into the working paper, would be required in April - I don't see this happening although it is a small and actually useful feature. One stumbling block is that whenever anything is brought up for IOStreams there is this major moan "can't we have something better?" but nobody comes forward with something better... – Dietmar Kühl Oct 24 '12 at 23:36
  • 9
    I am implementing something based on this, but I really need a working `tellg()` / `tellp()` for any `istream` / `ostream` instances that wrap the `membuf`. I've seen some indication that I need to implement `seekoff` and `seekpos`, but no solid working examples of that being done. Any insight? – Brian McFarland Jun 30 '15 at 20:08
  • Note that this code has a memory leak! I edited it accordingly. – Andrew Jan 30 '17 at 03:30
  • 5
    @Andrew: the code doesn't allocate any memory, hence it doesn't deallocate any. The change is specifically wrong when `data` happens to point to a string literal. – Dietmar Kühl Jan 30 '17 at 03:50
  • And what if it doesn't? That needs to be taken into consideration somehow. This code is incomplete. – Andrew Jan 30 '17 at 13:16
  • 6
    @Andrew: whoever allocates `data` (if it is allocated) is responsible for deallocating it. Since the code snippet doesn't show where the data comes from it shall not assume and do something! The code is clearly incomplete as there is no declaration of `data` or `size`. For the question and the answer the source of these doesn't matter nor does it matter what happens to them after they have been used. – Dietmar Kühl Jan 30 '17 at 13:39
  • That's not quite true here. When you pass a raw pointer of buffer data into a memory structure (particularly a buffer...), you at the very least need to make it clear whether that memory structure is being handed over that data to manage, if not delete it. You could have different constructors: one for a string, one for a pointer, or perhaps you could have static handler functions, but such a struct as this is very ambiguous and therefore deceptive. – Andrew Jan 30 '17 at 13:44
  • Compounding this is the fact that it inherits from an unfamiliar class. – Andrew Jan 30 '17 at 13:45
  • 3
    @Andrew: I'm confused - what class is "unfamiliar"? `std::streambuf` is the only base class here which is, clearly, a standard class (hence the `std::` prefix). How is that unfamiliar? For the purpose of looking at memory to read from the class is exactly as it is meant to be. ... as is the example. If you don't like the answer feel free to downvote and/or create your own answer. – Dietmar Kühl Jan 30 '17 at 13:49
  • Someone should not have to know all the inner workings of streambuf in order to use this solution. That defeats the purpose. – Andrew Jan 30 '17 at 13:55
  • 7
    @Andrew: does it defeat the purpose? The short snippet showed all the relevant bits needed to implement a production-ready version of the class. Surely, the production-ready version of the code comes with documentation, test-cases, etc. I don't think an answer needs to cover everything it using. – Dietmar Kühl Jan 30 '17 at 13:59
  • 2
    Mind if we extend this answer by an implementation for `seekpos` and `seekoff`? Took me a while to figure out the right way and could help other people looking for the same. – Jan Michael Auer Sep 06 '17 at 07:12
  • 3
    @Andrew : passing ownership of a buffer by just passing a raw pointer is not really a good idea in 2017. In that case you just pass a std::unique_ptr &&. Passing a raw const ptr suggests you do not pass ownership, at least in 2017 code. – galinette Nov 29 '17 at 12:56
  • @JanMichaelAuer Please go ahead - I'd love to see what you came up with. – Lightness Races in Orbit Nov 20 '18 at 17:31
  • FWIW after implementing `seekoff` exactly as laid out on cppreference.com (and `seekpos` too) then it worked nicely. Without those, seeks would always fail, but everything else worked. – Lightness Races in Orbit Nov 21 '18 at 10:29
  • @Andrew: We can put a comment on the class declaration that tells users the memory is non-owned by the class, though I think it's kind of obvious TBH, since that's _the entire purpose_ of the class. – Lightness Races in Orbit Feb 20 '19 at 13:45
  • @LightnessRacesinOrbit would it be possible to add the seekoff/seekpos code to the answer and make it more complete? otherwise I imagine more people will run into issues with buffer seeks and it avoids an external dependency (though I imagine cppreference is not going places soon) – StarShine Nov 23 '20 at 08:53
  • I mean, `strstreambuf` has been deprecated for 22 years but otoh... it's still there after 22 years: `strstreambuf(start, length)`. It's exactly the same as the `membuf` here. See https://en.cppreference.com/w/cpp/io/strstreambuf and also https://stackoverflow.com/questions/11890803/was-strstream-ever-part-of-the-standard. – Jason C Dec 09 '20 at 17:36
1

The only way would be to subclass std::istream (which also requires subclassing std::streambuf) to create your own stream class that reads from constant memory.

It's not as easy as it sounds because the the C++ standard library stream classes are pretty messy and badly designed. I don't think it's worth it unless you need it to scale a lot.

user1610015
  • 6,561
  • 2
  • 15
  • 18