shared_ptr to variable length struct

Question

How can I combine the variable length struct idiom

struct Data
{
  std::size_t size;
  char data[];
};

with the make_shared idiom which does essentially the same thing so that I can end up with a shared_ptr to one contiguous block of memory which contains the ref count structure header and structure data.

i.e. something like

// allocate an extra 30 bytes for the data storage
shared_ptr<Data> ptr = allocate_shared<Data>( vls_allocator(30) );

You're using C++. Simply define a constructor for `Data` that takes in a size to allocate for the array, then do `shared_ptr ptr = shared_ptr(new Data(30))`. `struct` in C++ just means `class` with a default of `public` instead of `private` for members. — aruisdante, Mar 21 '14 at 01:07
Note flexible arrays are an extension, I am guessing you are using gcc or clang. — Shafik Yaghmour, Mar 21 '14 at 01:07
Adding a constructor does nothing here. By the time the constructor is called the memory is already allocated and it's too late to do anything. — Michael Marcin, Mar 21 '14 at 01:10
Yes flexible arrays (thanks for reminding me of the name) is an extension, a widely available and extremely useful one. — Michael Marcin, Mar 21 '14 at 01:11
Wouldn't you simply do the allocation in the instantiation-list? I.E. `Data(size_t size):size(size), data(vls_allocator(size)){}`? — aruisdante, Mar 21 '14 at 01:11
@aruisdante that won't work, see my [answer here](http://stackoverflow.com/a/20221073/1708801) for some details on flexible arrays. — Shafik Yaghmour, Mar 21 '14 at 01:11
Hmmmm is there a reason why something like this does not work for you?: `std::shared_ptr sp1( static_cast( malloc( sizeof(Data) + sizeof(char)*20 ) ), free ) ;` — Shafik Yaghmour, Mar 21 '14 at 01:33
There are 3 blocks of memory here. The shared_ptr ref_count memory, the Data struct header memory, and the Data struct data memory. This combines the latter 2 but still requires the shared_ptr to allocate it's ref count separately. The question is basically how to do this + make_shared. — Michael Marcin, Mar 21 '14 at 01:39

score 1 · Accepted Answer · answered Mar 21 '14 at 01:14

1

You can achieve this using boosts intrusive shared pointers, (Not sure if this is supported directly by C++11).

http://www.boost.org/doc/libs/1_55_0/libs/smart_ptr/intrusive_ptr.html

You'd need to create a larger struct with the reference count in it though and your pointers would point at the laregr structure rather than its contained Data member.

You also need to hook up your deallocation function into to these pointers.

NOTE: I suspect there are ways to get a shared pointer into the Data member, rather than the wrapper - but last time i did that with boost shared_ptr code it required some interesting hacks.

answered Mar 21 '14 at 01:14

Michael Anderson

70,661
7
134
187

Indeed this is not a bad option, the thread safe reference counter of shared_ptr may not be necessary in this case I'll have to think on it more. – Michael Marcin Mar 21 '14 at 01:20
You can still use a thread safe counter I think - but you'll need to check the docs.. The real loss is the loss of weak_ptr support. (I don't think there's an intrusive variant that supports that .. infact I dont think it can be done sanely with the memory layout you want.( – Michael Anderson Mar 21 '14 at 01:58
I determined that I do need the thread safe atomic counter but combining intrusive_ptr with the atomic usage described [in the boost atomic documentation](http://www.boost.org/doc/libs/1_55_0/doc/html/atomic/usage_examples.html#boost_atomic.usage_examples.example_reference_counters) is a solution – Michael Marcin Mar 21 '14 at 18:11

score 1 · Answer 2 · answered Apr 17 '20 at 15:19

This is an important requirement in my opinion especially when writing code which must interact with or provide interfaces to low level system functionality (outside the world of modern C++ but still compliant with modern code).

There are two parts to the solution. First allocating the buffer as a smart pointer and secondly casting it to the right type in a safe way which passes the code analysis/guideline checkers.

The second part is generally available with "std::reinterpret_pointer_cast<T>" however this will not accept a cast from any array type without spewing out warnings=errors from the compiler or guideline checkers.

The first part which is the missing link until C++20 is to create a safe shared_ptr to a buffer of variable length, for example "shared_ptr(size_t length)" created by "std::make_shared<T[]>(size_t)" but sadly forgotten in the compiler/standard library versions before C++20 (even though they did implement "make_unique<T>(size_t)" and the request for change for the same in make_shared has been around for years).

I stumbled upon this macro which when combined with std::reinterpret_pointer_cast gives you a nice "polyfill" to achieve the same result in clean code until C++20 is generally available.

// C++20 polyfill for missing "make_shared<T[]>(size_t size)" overload.
template<typename T>
inline std::shared_ptr<T> make_shared_array(size_t bufferSize)
{
    return std::shared_ptr<T>(new T[bufferSize], [](T* memory) { delete[] memory; });
}

// Creates a smart pointer to a type backed by a variable length buffer, e.g. system structures.
template<typename T>
inline std::shared_ptr<T> CreateSharedBuffer(size_t byteSize)
{
    return std::reinterpret_pointer_cast<T>(make_shared_array<uint8_t>(byteSize));
}

Example:

size_t privilegesSize = sizeof(TOKEN_PRIVILEGES) + (sizeof(LUID_AND_ATTRIBUTES) * privilegesCount);
auto tokenPrivileges = CreateSharedBuffer<TOKEN_PRIVILEGES>(privilegesSize);

The only side effect is apparently a second memory access will occur during allocation but given C++20 is around the corner it's probably more efficient and robust for your solution to get running now with clean robust code than worrying about an extra processor cycle or two for what will be a short period of time.

All you have to do when C++20 is available is delete the "polyfill" template and rename or replace the call to it from "make_shared_array" to "make_shared" (with the new C++20 overload). Optionally ditch the whole CreateSharedBuffer macro if the C++20 solution below is acceptable:

auto data = std::reinterpret_cast<Data>(std::make_shared<uint8_t>(sizeof(Data) + (sizeof(char) * 30)));

shared_ptr to variable length struct

2 Answers2