40

Apparently boost::asio::async_read doesn't like strings, as the only overload of boost::asio::buffer allows me to create const_buffers, so I'm stuck with reading everything into a streambuf.
Now I want to copy the contents of the streambuf into a string, but it apparently only supports writing to char* (sgetn()), creating an istream with the streambuf and using getline().

Is there any other way to create a string with the streambufs contents without excessive copying?

tstenner
  • 10,080
  • 10
  • 57
  • 92

11 Answers11

51

I don't know whether it counts as "excessive copying", but you can use a stringstream:

std::ostringstream ss;
ss << someStreamBuf;
std::string s = ss.str();

Like, to read everything from stdin into a string, do

std::ostringstream ss;
ss << std::cin.rdbuf();
std::string s = ss.str();

Alternatively, you may also use a istreambuf_iterator. You will have to measure whether this or the above way is faster - i don't know.

std::string s((istreambuf_iterator<char>(someStreamBuf)), 
               istreambuf_iterator<char>());

Note that someStreamBuf above is meant to represent a streambuf*, so take its address as appropriate. Also note the additional parentheses around the first argument in the last example, so that it doesn't interpret it as a function declaration returning a string and taking an iterator and another function pointer ("most vexing parse").

Johannes Schaub - litb
  • 496,577
  • 130
  • 894
  • 1,212
  • 3
    Thanks, istreambuf_iterator was what I've been looking for. – tstenner May 18 '09 at 15:28
  • 1
    Here's something strange. I can't assume why, but istreambuf_iterator cuts last symbols (if it's more than 20 symbols in last line). Is there any ideas of why could it be? – Ben Usman Feb 09 '10 at 14:21
  • ss << someStreamBuf copies the 11 bytes '0xFFFFFFFFF' which represent the address of the streambuf in memory. thanks. very helpful :D – Alex Kremer Jul 17 '13 at 19:36
  • 1
    @AlexKremer obviously `someStreamBufPointer` was meant – sehe Sep 23 '15 at 21:31
  • Since C++11, the "most vexing parse" problem can also be avoided by `istreambuf_iterator{someStreamBuf}`. – MSalters Nov 05 '18 at 10:45
50

It's really buried in the docs...

Given boost::asio::streambuf b, with size_t buf_size ...

boost::asio::streambuf::const_buffers_type bufs = b.data();
std::string str(boost::asio::buffers_begin(bufs),
                boost::asio::buffers_begin(bufs) + buf_size);
Christian Severin
  • 1,793
  • 19
  • 38
Sean DeNigris
  • 6,306
  • 1
  • 31
  • 37
  • 5
    This works well! You can use `b.size()` instead of `buf_size` if you want the whole thing put into the string. – Malvineous May 18 '13 at 11:55
  • 1
    It is supposed that the answer of tstenner above where he says that it is considered dirty and won't work with 'normal streambufs' apply to this answer or this way is considered safe? – ChrisPeterson Feb 11 '15 at 16:10
24

Another possibility with boost::asio::streambuf is to use boost::asio::buffer_cast<const char*>() in conjunction with boost::asio::streambuf::data() and boost::asio::streambuf::consume() like this:

const char* header=boost::asio::buffer_cast<const char*>(readbuffer.data());
//Do stuff with header, maybe construct a std::string with std::string(header,header+length)
readbuffer.consume(length);

This won't work with normal streambufs and might be considered dirty, but it seems to be the fastest way of doing it.

tstenner
  • 10,080
  • 10
  • 57
  • 92
  • where does length come from? – Trevor Hickey Apr 22 '13 at 06:45
  • It's the number of bytes you took from the stream. See http://www.boost.org/doc/libs/1_53_0/doc/html/boost_asio/reference/basic_streambuf/consume.html – tstenner Apr 22 '13 at 13:26
  • What you mean with 'normal streambufs' ? The answer of Sean DeNigris bellow based on the documentation wouldn't be valid? – ChrisPeterson Feb 11 '15 at 16:06
  • 2
    Just as a note to people seeing this now, this will only work if your stream has a single buffer object allocated and populated. If the stream has more than one buffer, you're screwed this way. What you'll need to do and what you should do for this to be guaranteed to work is to get an iterator to the buffer(s) and then cast the iterators. You can get these iterators by accessing buffer->data().begin(). –  May 01 '15 at 02:33
  • I could not find any infromation on if and how boost::asio::steambuf allocates multiple buffer objects. Could you tell me what the requierements are for streambuf to return more than one buffer. – eclipse Aug 26 '15 at 21:14
  • 1
    @eclipse You gotta tag people! I didn't see your comment, and I actually came back here and found my own comment because I forgot about this gotcha lol. Anyway the streambuf internally will allocate multiple internal buffers which the single streambuf spans. To be safe, you absolutely must use the iterators. Using a straight up cast like this will give you a pointer that will lead to undefined behavior, whether it's that the ordering of data is messed up, or the length you use to read from the pointer is out of bounds. –  Feb 09 '16 at 14:11
16

For boost::asio::streambuf you may find a solution like this:

    boost::asio::streambuf buf;
    /*put data into buf*/

    std::istream is(&buf);
    std::string line;
    std::getline(is, line);

Print out the string :

    std::cout << line << std::endl;

You may find here: http://www.boost.org/doc/libs/1_49_0/doc/html/boost_asio/reference/async_read_until/overload3.html

iericzhou
  • 620
  • 8
  • 11
  • This answer is definitely better if you're using `async_read_until` because that call can return more than one line of data, and this answer correctly leaves the extra data in the `streambuf`. – Jim Hunziker Jan 12 '17 at 14:22
2

One can also obtain the characters from asio::streambuf using std::basic_streambuf::sgetn:

asio::streambuf in;
// ...
char cbuf[in.size()+1]; int rc = in.sgetn (cbuf, sizeof cbuf); cbuf[rc] = 0;
std::string str (cbuf, rc);
ArtemGr
  • 11,684
  • 3
  • 52
  • 85
1

I didn't see an existing answer for reading exactly n chars into a std::stringstream, so here is how that can be done:

std::stringstream ss;
boost::asio::streambuf sb;
const auto len = 10;

std::copy_n(boost::asio::buffers_begin(sb.data()), len,
            std::ostream_iterator<decltype(ss)::char_type>(ss));

Compiler explorer

Zitrax
  • 19,036
  • 20
  • 88
  • 110
1

The reason you can only create const_buffer from std::string is because std::string explicitly doesn't support direct pointer-based writing in its contract. You could do something evil like resize your string to a certain size, then const_cast the constness from c_str() and treat it like a raw char* buffer, but that's very naughty and will get you in trouble someday.

I use std::vector for my buffers because as long as the vector doesn't resize (or you are careful to deal with resizing), you can do direct pointer writing just fine. If I need some of the data as a std::string, I have to copy it out, but the way I deal with my read buffers, anything that needs to last beyond the read callback needs to be copied out regardless.

user121826
  • 51
  • 3
0

A simpler answer would be to convert it in std::string and manipulate it some what like this

 std::string buffer_to_string(const boost::asio::streambuf &buffer)
 {
  using boost::asio::buffers_begin;
  auto bufs = buffer.data();
  std::string result(buffers_begin(bufs), buffers_begin(bufs) + buffer.size());
 return result;
}

Giving a very concise code for the task.

DevMac
  • 131
  • 1
  • 2
  • 9
0

I mostly don't like answers that say "You don't want X, you want Y instead and here's how to do Y" but in this instance I'm pretty sure I know what tstenner wanted.

In Boost 1.66, the dynamic string buffer type was added so async_read can directly resize and write to a string buffer.

tstenner
  • 10,080
  • 10
  • 57
  • 92
-1

I tested the first answer and got a compiler error when compiling using "g++ -std=c++11" What worked for me was:

        #include <string>
        #include <boost/asio.hpp>
        #include <sstream>           
        //other code ...
        boost::asio::streambuf response;
        //more code
        std::ostringstream sline;
        sline << &response; //need '&' or you a compiler error
        std::string line = sline.str(); 

This compiled and ran.

-2

I think it's more like:


streambuf.commit( number_of_bytes_read );

istream istr( &streambuf );
string s;
istr >> s;

I haven't looked into the basic_streambuf code, but I believe that should be just one copy into the string.

Nikolai Fetissov
  • 82,306
  • 11
  • 110
  • 171