0

I am creating an interprocess communication scheme using boost::interprocess. I want my shared memory area to contain some std::vectors and am doing this based on the example provided here.

To communicate, I plan to put a struct like the following in the shared memory (not even attempted to compile this yet btw):

// Define an STL compatible allocator of doubles that allocates from the managed_shared_memory.
// This allocator will allow placing stl containers in the segment
typedef allocator<doublereal, boost::interprocess::managed_shared_memory::segment_manager>  ShmemAllocator_doublereal;

// Alias a vector that uses the previous STL-like allocator so that it
// allocates its values from the segment
typedef vector<doublereal, ShmemAllocator_doublereal> shmVector_doublereal;

// Define an STL compatible allocator for uint32_t in shared memory
typedef allocator<uint32_t, boost::interprocess::managed_shared_memory::segment_manager>  ShmemAllocator_uint32_t;

// Shared memory vector for uint32_t type for lables
typedef vector<uint32_t, ShmemAllocator_uint32_t> shmVector_uint32_t;


struct shared_memory_buffer {
    // Server initialized with one to start, client must wait for it to be released
    shared_memory_buffer(int nnodes, boost::interprocess::managed_shared_memory segment): server(1), client(0){

        // Initialize shared memory STL-compatible allocators
        const ShmemAllocator_doublereal alloc_inst_d (segment.get_segment_manager());
        const ShmemAllocator_uint32_t alloc_inst_u (segment.get_segment_manager());

        // Construct the data vectors
        motion_data = segment.construct<shmVector_doublereal>("motion_data_vector")(alloc_inst_d);

        force_data = segment.construct<shmVector_doublereal>("force_data_vector")(alloc_inst_d);

        labels = segment.construct<shmVector_uint32_t>("label_vector")(alloc_inst_u);

        // position, velocity and acceleration in 6dof
        motion_data.resize (nnodes * 6  * 3);

        // force in 6dof
        force_data.resize (nnodes * 6);

        // n node labels
        labels.resize (nnodes);

    }

    // convenience function to get the motion data vector from the shared
    // memory segment
    shmVector_doublereal<doublereal, ShmemAllocator_doublereal> * getMotionData (boost::interprocess::managed_shared_memory segment) {
        return segment.find<shmVector_doublereal>("motion_data_vector").first;
    }

    // convenience function to get the force data vector from the shared
    // memory segment
    shmVector_doublereal<doublereal, ShmemAllocator_doublereal> * getMotionData (boost::interprocess::managed_shared_memory segment) {
        return segment.find<shmVector_doublereal>("force_data_vector").first;
    }

    // convenience function to get the buffer from the shared
    // memory segment
    shmVector_doublereal<doublereal, shmMemAllocator> * getLabels (boost::interprocess::managed_shared_memory segment) {
        return segment.find<shmVector_uint32_t>("label_vector").first;
    }

    // semaphores for read/write control
    boost::interprocess::interprocess_semaphore server, client;

    // buffer for motion data
    shmVector_doublereal<doublereal, shmMemAllocator> *motion_data;

    // buffer for force data
    shmVector_doublereal<doublereal, shmMemAllocator> *force_data;

    // buffer for label data
    shmVector_uint32_t<uint32_t, shmMemAllocator> *labels;

    //doublereal time;

};

My problem is, how can I know how much memory to allocate when creating the shared memory, e.g. with 

managed_shared_memory segment(create_only, "MySharedMemory", 65536);

I therefore want to know/estimate how much memory in total must be allocated to hold the entire structure, which contains stl vectors, in the shared memory area.

At run time, before shared memory allocation I will always know how many elements will be in the vectors (see the struct constructor to give an idea of this), but not at compile time, which is one reason I'm using vectors. I could just allocate what I deem to be quite a lot, based on how I think people might use this, but no doubt some day someone will use too many nodes and will have difficulties.

crobar
  • 2,810
  • 4
  • 28
  • 46
  • not sure if I understand the problem, why cant you replace the `65536` with whatever size you need (depending on the size of your vectors) at runtime. – 463035818_is_not_an_ai Jul 27 '17 at 10:04
  • @tobi303 I want to do this, the question is how to decide/calculate what the new number should be. – crobar Jul 27 '17 at 10:06
  • you wrote "before shared memory allocation I will always know how many elements will be in the vectors" ... then just add up the sizes to get the total size – 463035818_is_not_an_ai Jul 27 '17 at 10:07
  • @tobi303 what about the structure itself and stl container overhead? – crobar Jul 27 '17 at 10:08
  • so you actually want to know how much memory a vector occupies in total? – 463035818_is_not_an_ai Jul 27 '17 at 10:09
  • @tobi303, I want to know how much memory in total must be allocated to hold the entire structure, which contains stl vectors, in the shared memory area. This is exactly what the question is. – crobar Jul 27 '17 at 10:11
  • yeah now I understand the question, but I dont know the answer ;) – 463035818_is_not_an_ai Jul 27 '17 at 10:12
  • 1
    You're probably going to be allocating in page sizes in reality anyway, so just round up a page or two? Unless you have a lot of these or no memory, is it worth getting an exact size? – Joe Jul 27 '17 at 11:14
  • @Joe, I will accept an answer which explains how to do this and explains why this would be plenty of memory for the struct. I really just want to make sure I allocate enough memory, but accounting for someone working on a problem which takes them close to the limit of available memory (i.e. not massively overallocating). I also want this to be portable, and I get the impression [page size is not portable](https://stackoverflow.com/questions/3351940/detecting-the-memory-page-size) – crobar Jul 27 '17 at 11:18
  • If it's not 4K I'll eat my socks. – Joe Jul 27 '17 at 11:47
  • 1
    @Joe but I really need this to run on [UltraSPARC Architecture 2007](https://en.wikipedia.org/wiki/Page_(computer_memory)#Multiple_page_sizes)! Only joking, you are right, and since I'm requiring boost, I can also use `mapped_region::get_page_size()` mentioned in the link I gave. But is it pretty safe to allocate one page for the structure (plus an estimate based on calculating the array sizes from the number of nodes)? I have no idea what size structure stuff is in memory, although I get the impression it is very small. – crobar Jul 27 '17 at 12:16
  • It should be, it certainly won't be a whole page, so if you round up to the nearest page and add one, I'd be highly surprised if you're not solid. As to how to calculate the entire size of the overhead, perhaps this helps? https://stackoverflow.com/questions/557997/what-is-the-overhead-cost-of-an-empty-vector – Joe Jul 27 '17 at 12:27

0 Answers0