1

I have a pointer to a const *char buffer as well as it's length, and am trying to use an API (in this case, the AWS S3 C++ upload request) that accepts an object of type:

std::basic_iostream <char, std::char_traits <char>>

Is there a simple standard C++11 way to convert my buffer into a compatible stream, preferably without actually copying over the memory?

nbubis
  • 2,304
  • 5
  • 31
  • 46
  • When you say `I have a pointer to a const *char buffer` do you really mean that you have a pointer to a `const char* buffer[N]`, or did you intend to say that you have a pointer to a (zero terminated?) buffer of characters? – eerorika Jan 03 '17 at 17:07
  • @user2079303 - The latter. It's a binary data buffer so it's not necessarily zero terminated. – nbubis Jan 03 '17 at 17:08
  • [This](http://stackoverflow.com/questions/2079912/simpler-way-to-create-a-c-memorystream-from-char-size-t-without-copying-t) question looks related to me – UKMonkey Jan 03 '17 at 17:11
  • @UKMonkey - Indeed, however, one solution involves boost which I can't currently use, and the second (pubsetbuf) doesn't work for const char. – nbubis Jan 03 '17 at 17:23
  • [`std::strstream`](http://en.cppreference.com/w/cpp/io/strstream) can do that, but it's deprecated. The non-deprecated counterpart is [`std::stringstream`](http://en.cppreference.com/w/cpp/io/basic_stringstream), but it can't work off a client-provided buffer - it always makes a copy. – Igor Tandetnik Jan 03 '17 at 17:26

3 Answers3

4

Thanks to Igor's comment, this seems to work:

func(const * char buffer, std::size_t buffersize)
{     
    auto sstream = std::make_shared<std::stringstream>();
    sstream->write(buffer, buffersize);
    ...
    uploadRequest.SetBody(sstream);     
    ....
nbubis
  • 2,304
  • 5
  • 31
  • 46
  • 1
    @MarkR - Probably best to add this as an answer. – nbubis Sep 08 '17 at 18:25
  • I thought this was the answer I, too, was looking for. But it doesn't compile: `no member named 'write' in 'std::__1::basic_istringstream, std::__1::allocator >'` – ndtreviv Feb 22 '18 at 11:06
  • Actually, it does compile if you use the `std::stringstream::out` flag when calling `make_shared`. – ndtreviv Feb 22 '18 at 11:35
  • FYI It helped me answer my question: https://stackoverflow.com/questions/48666549/upload-uint8-t-buffer-to-aws-s3-without-going-via-filesystem/48926299#48926299 – ndtreviv Feb 22 '18 at 11:36
  • 1
    Doesn't `->write` do a copy of the data, though? I'm still looking for a copyless solution, will post if I succeed. – hfingler Jan 28 '20 at 23:18
2

As a fairly obvious corollary to your solution, you can create an empty basic_iostream with code like this. This example creates a 0-byte pseudo-directory S3 key: 

Aws::S3::Model::PutObjectRequest object_request; 
// dirName ends in /.             
object_request.WithBucket(bucketName).WithKey(dirName); 
// Create an empty input stream to create the 0-byte directory file. 
auto empty_sstream = std::make_shared<std::stringstream>();    
object_request.SetBody(empty_sstream); 
auto put_object_outcome = s3_client->PutObject(object_request);
Mark R
  • 251
  • 3
  • 12
1

If you do not want to make a copy of your data, and assuming using boost is an option, you can use basic_bufferstream from boost:

#include <boost/interprocess/streams/bufferstream.hpp>
char* buf = nullptr; // get your buffer
size_t length = 0; 
auto input = Aws::MakeShared<boost::interprocess::basic_bufferstream<char>> (
             "PutObjectInputStream",                                         
             buf,                                                            
             length);

Then your s3 client can use it:

 Aws::S3::Model::PutObjectRequest req;
 req.WithBucket(bucket).WithKey(key);
 req.SetBody(input);
 s3.PutObject(req);
Yefu
  • 161
  • 2