36

I couldn't find anything ready-made, so I came up with:

class membuf : public basic_streambuf<char>
{
public:
  membuf(char* p, size_t n) {
    setg(p, p, p + n);
    setp(p, p + n);
  }
}

Usage:

char *mybuffer;
size_t length;
// ... allocate "mybuffer", put data into it, set "length"

membuf mb(mybuffer, length);
istream reader(&mb);
// use "reader"

I know of stringstream, but it doesn't seem to be able to work with binary data of given length.

Am I inventing my own wheel here?

EDIT

  • It must not copy the input data, just create something that will iterate over the data.
  • It must be portable - at least it should work both under gcc and MSVC.
Community
  • 1
  • 1
Marcin Seredynski
  • 7,057
  • 3
  • 22
  • 29

4 Answers4

33

I'm assuming that your input data is binary (not text), and that you want to extract chunks of binary data from it. All without making a copy of your input data.

You can combine boost::iostreams::basic_array_source and boost::iostreams::stream_buffer (from Boost.Iostreams) with boost::archive::binary_iarchive (from Boost.Serialization) to be able to use convenient extraction >> operators to read chunks of binary data.

#include <stdint.h>
#include <iostream>
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/archive/binary_iarchive.hpp>

int main()
{
    uint16_t data[] = {1234, 5678};
    char* dataPtr = (char*)&data;

    typedef boost::iostreams::basic_array_source<char> Device;
    boost::iostreams::stream_buffer<Device> buffer(dataPtr, sizeof(data));
    boost::archive::binary_iarchive archive(buffer, boost::archive::no_header);

    uint16_t word1, word2;
    archive >> word1 >> word2;
    std::cout << word1 << "," << word2 << std::endl;
    return 0;
}

With GCC 4.4.1 on AMD64, it outputs:

1234,5678

Boost.Serialization is very powerful and knows how to serialize all basic types, strings, and even STL containers. You can easily make your types serializable. See the documentation. Hidden somewhere in the Boost.Serialization sources is an example of a portable binary archive that knows how to perform the proper swapping for your machine's endianness. This might be useful to you as well.

If you don't need the fanciness of Boost.Serialization and are happy to read the binary data in an fread()-type fashion, you can use basic_array_source in a simpler way:

#include <stdint.h>
#include <iostream>
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream.hpp>

int main()
{
    uint16_t data[] = {1234, 5678};
    char* dataPtr = (char*)&data;

    typedef boost::iostreams::basic_array_source<char> Device;
    boost::iostreams::stream<Device> stream(dataPtr, sizeof(data));

    uint16_t word1, word2;
    stream.read((char*)&word1, sizeof(word1));
    stream.read((char*)&word2, sizeof(word2));
    std::cout << word1 << "," << word2 << std::endl;

    return 0;
}

I get the same output with this program.

Emile Cormier
  • 28,391
  • 15
  • 94
  • 122
  • Great stuff, Emile. Maybe it's not "simpler", but it's more generic for sure. Thank you! – Marcin Seredynski Jan 17 '10 at 08:13
  • 1
    The reason only fstream objects have the concept of binary is that text and binary modes are exactly the same; apart from how they handle end of line. In terms of actions for stream operations they are no different. The way you are using the stream operators to try and read integers from a char stream is not how the stream operators are supposed to work. – Martin York Jan 17 '10 at 09:44
  • 2
    I just want to register disagreement with Loki. The `<<` and `>>` operators are not "supposed to work" the same when used on other data types. If you go in that direction, you'd have no choice but to conclude that iostreams using them as "stream operators" is broken, because that's not how the **bitshift operators** (yes, that's their true name) are supposed to work. – Ben Voigt Sep 04 '13 at 22:33
6

I'm not sure what you need, but does this do what you want?

char *mybuffer;
size_t length;
// allocate, fill, set length, as before

std::string data(mybuffer, length);
std::istringstream mb(data);
//use mb
crmoore
  • 399
  • 3
  • 7
  • 2
    No, this will truncate the string to the first occurrence of 0x00 byte in the buffer. I specifically need to create a fixed-length stream that I can read binary data from, when I have this data in a known place in memory. – Marcin Seredynski Jan 17 '10 at 04:30
  • 4
    I don't see why that would handle NUL bytes specially. Notice length being passed to std::string constructor separately. – Tronic Jan 17 '10 at 04:45
  • 2
    Are you sure? The string constructor that takes (char*, size_t) does not treat the char* param as a c-style (null-terminated) string. It uses the length you give it. – crmoore Jan 17 '10 at 04:49
  • 2
    Yes, sorry, you're right. I was misled by VS debugger visualizer. It truncated the string on NUL character. The remainder of data is put into string. – Marcin Seredynski Jan 17 '10 at 04:59
  • 2
    While your approach gives me what I need, it does have a side effect of the input data being copied into the `data` string. I would like to avoid that. Sorry for not making this clear earlier. – Marcin Seredynski Jan 17 '10 at 05:04
4

The standard stream buffer has this functionality.
Create a stream. Gets its buffer then over-ride it.

#include <sstream>
#include <iostream>
#include <algorithm>
#include <iterator>

int main()
{
    // Your imaginary buffer
    char    buffer[]    = "A large buffer we don't want to copy but use in a stream";

    // An ordinary stream.
    std::stringstream   str;

    // Get the streams buffer object. Reset the actual buffer being used.
    str.rdbuf()->pubsetbuf(buffer,sizeof(buffer));

    std::copy(std::istreambuf_iterator<char>(str),
              std::istreambuf_iterator<char>(),
              std::ostream_iterator<char>(std::cout)
             );
}
sbi
  • 219,715
  • 46
  • 258
  • 445
Martin York
  • 257,169
  • 86
  • 333
  • 562
  • Hmmm... I put this to test, and it doesn't seem to work under MSVC and its libs. It works on gcc, though. – Marcin Seredynski Jan 17 '10 at 05:50
  • Looks like MSVC's basic_streambuf::pubsetbuf doesn't do anything, at all. http://stackoverflow.com/questions/1494182/setting-the-internal-buffer-used-by-a-standard-stream-pubsetbuf – Marcin Seredynski Jan 17 '10 at 11:44
  • 4
    And it can't be even called a bug, as C++ standards don't define what is the expected behavior of calling `setbuf`: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1905.pdf - Appendix D.7.26 – Marcin Seredynski Jan 17 '10 at 12:02
  • @ybungalobill: Yes so. Implementation defined does not mean `undefined` there is a big difference. Implementation defined means the implementers have some lee way but it still works (as expected). – Martin York Dec 03 '10 at 20:10
  • 2
    @Martin: You are just fortunate it works on your implementation that defined it this way. Mine implementation, and @Marcin's too, doesn't implement it this way. Our implementation is still a standard conforming implementation. – Yakov Galka Dec 03 '10 at 20:13
  • @ybungalobill: You do realize that the section mentioned D.7.26 is for the deprecates strstreambuf not the stringstream_buffer stream buffer that we are using above. Read: 27.5.2.4.2 – Martin York Dec 03 '10 at 20:39
  • 2
    @Martin: @ybungalobill is correct. It's section 27.8.1.4 in the FCD, and it's still implementation-defined for all combinations of parameters except `setbuf(0, 0)`. – Ben Voigt Dec 09 '10 at 20:19
2

The questioner wanted something that doesn't copy the data, and his solution works fine. My contribution is to clean it up a little, so you can just create a single object that's an input stream for data in memory. I have tested this and it works.

class MemoryInputStream: public std::istream
    {
    public:
    MemoryInputStream(const uint8_t* aData,size_t aLength):
        std::istream(&m_buffer),
        m_buffer(aData,aLength)
        {
        rdbuf(&m_buffer); // reset the buffer after it has been properly constructed
        }

    private:
    class MemoryBuffer: public std::basic_streambuf<char>
        {
        public:
        MemoryBuffer(const uint8_t* aData,size_t aLength)
            {
            setg((char*)aData,(char*)aData,(char*)aData + aLength);
            }
        };

    MemoryBuffer m_buffer;
    };
Graham Asher
  • 1,648
  • 1
  • 24
  • 34
  • 3
    Damn - it doesn't work because seekg, etc., won't work by default - you have to override various streambuf functions. – Graham Asher May 16 '12 at 15:52