Check boost::interprocess::vector has enough capacity before writing?

Question

I am using a shared vector to share objects across memory:

using ShmemAllocator = bip::allocator<T, bip::managed_shared_memory::segment_manager>;
using MyVector = bip::vector<T, ShmemAllocator>;

bip::permissions perm;
perm.set_unrestricted();
segment.reset(new bip::managed_shared_memory(bip::open_or_create, shared_memory_name, numBytes, 0, perm));

const ShmemAllocator alloc_inst(segment->get_segment_manager());
vec = segment->find_or_construct<MyVector>(shared_vector_name)(alloc_inst);

Note the vector is created within a managed_shared_memory object and this is created by specifying a number of bytes, not number of vector elements.

I then write elements to the vector:

int write(const std::vector<T>& vec)
{
    bip::scoped_lock<bip::named_mutex> lock(*sdc.mutex);

    for(const auto& item : vec)
    {
        sdc.vec->push_back(item);
    }

    sdc.cond_empty->notify_all();
}

What is the safest way to check whether I have enough space to write all my elements, prior to writing? I would really like to avoid simply assigning a large number of bytes and hoping I never hit it!

@VTT that might fail. Checking [capacity()](http://en.cppreference.com/w/cpp/container/vector/capacity) is closer already — sehe, May 16 '18 at 22:58
@sehe I didn't see your comment and implemented what VTT is probably suggesting, whereby I try doing a reserve(vec.size() + newVec.size()) and if that fails, I catch and return false. Is capacity() the correct way of implementing what I ended up doing? — user997112, May 18 '18 at 09:23
`capacity()` is the right way to answer the bolded question _"What is the safest way to check whether I have enough space to write all my elements, prior to writing?"_. However the backstory suggests that you want to know ***"How do I make sure my memory segment is large enough to contain my data (e.g. a vector)?"*** which is what I answered in my question. — sehe, May 18 '18 at 10:06

score 0 · Answer 1 · answered May 16 '18 at 22:56

Note the vector is created within a managed_shared_memory object and this is created by specifying a number of bytes, not number of vector elements

There's no straight formula as the overhead of the segment manager can be significant and depends on allocation patterns: in this older answer I demonstrate this quite visually:

With that in mind, you can probably do some testing on the intended platform and workout the actual size required.

However, you can also forget about it, and remember it's all virtual memory anyways. Just reserve a TB or three, and nothing should ever get mapped/committed unless you those pages.

If you want to use managed_mapped_file you can happily use ftruncate to create a Very Large™ sparse file (i.e. with no blocks actually allocated) and use it in the very same way.

Not all filesystems support sparse files, but all mainstream filesystems do (NTFS, ext2/3/4 etc.). Usually if sparse files aren't supported, mmap isn't supported either (think network filesystems or highlevel fuse drivers)

Check boost::interprocess::vector has enough capacity before writing?

1 Answers1