Modern C++ idiom for allocating / deallocating an I/O buffer

Question

For I/O work, I need to read N bytes into a buffer. N is known at run time (not compile time). The buffer size will never change. The buffer is passed to other routines to compress, encrypt, etc: it's just a sequence of bytes, nothing higher than that.

In C, I would allocate the buffer with malloc and then free it when I'm done. However, my code is modern C++, certainly no mallocs in place, and very few raw new and delete: I'm making heavy use of RAII and shared_ptr. None of those techniques seem appropriate for this buffer, however. It's just a fixed length buffer of bytes, to receive I/O and make its contents available.

Is there a modern C++ idiom to this elegantly? Or, for this aspect, should I just stick with good ol' malloc?

Answer 1

Basically, you have two main C++-way choices:

• std::vector
• std::unique_ptr

I'd prefer the second, since you don't need all the automatic resizing stuff in std::vector, and you don't need a container - you need just a buffer.

std::unique_ptr has a specialization for dynamic arrays: std::unique_ptr<int[]> will call delete [] in it's destructor, and will provide you with appropriate operator [].

If you want the code:

std::unique_ptr<char[]> buffer(new char [size]);
some_io_function(buffer.get(), size); // get() returnes raw pointer

Unfortunatelly, it doesn't have a way to retrieve the size of the buffer, so you'll have to store it in a variable. If it confuses you, then std::vector will do the work:

std::vector<char> buffer(size);
some_io_function(buffer.data(), buffer.size()); // data() returnes raw pointer

If you want to pass the buffer around, it depends on how exactly you do it.

Consider the following case: the buffer is filled somewhere, then processed somewhere else, stored for some time, then written somewhere and destroyed. It happens that you never really need two places in the code to own the buffer, and you can simply std::move it from place to place. For this use case, std::unique_ptr will work perfectly, and will protect you from occasionally copying the buffer (while with std::vector you can copy it by mistake, and no error or warning will arise).

If, conversely, you need several places in the code to hold the same buffer (maybe it is filled / used / processed in more then one place simultaneously), you definitely need std::shared_ptr. Unfortunately, it does not have array-like specialization, so you'll have to pass appropriate deleter:

std::shared_ptr<char> buffer(new char[size], std::default_delete<char[]>());

The third option is if you really need to copy the buffer. Then, std::vector will be simpler. But, as I've already mentioned, I feel that it is not the best way. Also, you can always copy the buffer hold by std::unique_ptr or std::shared_ptr manually, which clearly documents your intention:

std::uniqure_ptr<char[]> buffer_copy(new char[size]);
std::copy(buffer.get(), buffer.get() + size, buffer_copy.get());

Answer 2

In C++14, there's a very syntactically clean way of achieving what you want:

size_t n = /* size of buffer */;
auto buf_ptr = std::make_unique<uint8_t[]>(n);
auto nr = ::read(STDIN_FILENO, buf_ptr.get(), n);
auto nw = ::write(STDOUT_FILENO, buf_ptr.get(), nr);
// etc.
// buffer is freed automatically when buf_ptr goes out of scope

Note that the above construct will value-initialize (zero out) the buffer. If you want to skip the initialization to save a few cycles, you'll have to use the slightly uglier form given by lisyarus:

std::unique_ptr<uint8_t[]> buf_ptr(new uint8_t[n]);

C++20 introduces std::make_unique_for_overwrite, which allows the non-initializing line above to be written more concisely as:

auto buf_ptr = std::make_unique_for_overwrite<uint8_t[]>(n);

Answer 3

Yes, easy:

std::vector<char> myBuffer(N);

Answer 4

I think its common to use a std::vector for this.

The benefits of using std::vector over an manually allocated char buffer are copy semantics (for passing to functions that wish to modify the data for their own purposes or when returning the data to a calling function).

Also a std::vector knows its own size reducing the number of parameters that need passing to processing functions and eliminating a source of bugs.

You have complete control over how the data is passed to other functions - either by reference or const reference as appropriate.

If you need to call an older c-style function with a plain char* and length you can easily do that too:

// pass by const reference to preserve data
void print_data(const std::vector<char>& buf)
{
    std::cout.fill('0');
    std::cout << "0x";
    for(auto c: buf)
        std::cout << std::setw(2) << std::hex << int(c);
    std::cout << '\n';
}

// pass by reference to modify data
void process_data(std::vector<char>& buf)
{
    for(auto& c: buf)
        c += 1;
}

// pass by copy to modify data for another purpose
void reinterpret_data(std::vector<char> buf)
{
    // original data not changed
    process_data(buf);
    print_data(buf);
}

void legacy_function(const char* buf, std::size_t length)
{
    // stuff
}

int main()
{
    std::ifstream ifs("file.txt");

    // 24 character contiguous buffer
    std::vector<char> buf(24);

    while(ifs.read(buf.data(), buf.size()))
    {
        // changes data (pass by reference)
        process_data(buf);

        // modifies data internally (pass by value)
        reinterpret_data(buf);

        // non-modifying function (pass by const ref)
        print_data(buf);

        legacy_function(buf.data(), buf.size());
    }
}

Answer 5

Use

std::vector<char> buffer(N)

If the size of the buffer never changes, you can use it as an array by doing this:

char * bufPtr = &buffer[0];

This will work in C++03 as well. See this comment https://stackoverflow.com/a/247764/1219722 for details on why this is safe.