0

I would like to pre-allocate sufficiently large amount of memory in a program before forking processes and then further allocate/use memory from this pool in the forked processes. I have come across some memory allocators like Bget, Boost etc but not able to understand how to use them.

Is there a simplest one out there which I can use like

poolhandle = poolallocate(pool_size)

Then in forked process use something like

ptr = allocatefromPool(poolhandle,no_of_bytes)

and then if I pass this pointer to another process through some IPC it should be accessible even in that process.

Can you point me in right direction ? If Boost is the way to go can you provide me an example on how to use it ?

lucent
  • 124
  • 8

1 Answers1

1

Honestly, the easiest way to do this is to use a memory-mapped file. Then you just do seek and write to save things, or seek and read to get things.

Beyond that, there is always the concept of actors where you don't share any memory at all, just send messages between actors, each of whom is the curator of their own datastore. This is simpler to program, especially with tools like ZeroMQ that make the interprocess and interthread messaging so simple.

If you combine the two ideas then each actor process has an area of the memory mapped file which they own, and you pass messages back and forth to tell them what to read or write.

See this question for more on memory-mapped files as shared memory Posix shared memory vs mapped files

Here is one about IPC which you will need for multiple processes to coordinate actions IPC vs domain sock vs named pipes

If you have large blocks of data to move around in messages then you would probably want to leave the data in place in the memory-mapped file and implement some form of lock-free sharing. There is lots of stuff you can Google using the keyword lock-free in conjunction with "shared memory" or "data structures".

Community
  • 1
  • 1
Michael Dillon
  • 31,973
  • 6
  • 70
  • 106
  • Hi Michael, Thanks for providing those inputs. Memory-mapped file is one option I am aware of but it is the requirement of my Task to use an readily available simple allocator or write one myself..basically some sort of heap/memory manager. I would rather like to explore the available options first and see if it meets the requirement. memory-mapped file i.e mmap I've heard could be slow as it involves couple of system calls ?(sorry I have used it but haven't validated this) – lucent Feb 17 '12 at 06:58
  • Are there any popular and simple heap manager/memory allocators/pools available that I can use ? Also boost doesn't actually pre-allocate the memory, does it? Basically I would like to pre-allocate a large chunk of memory and then allocate my structs,arrays,ints,chars..etc data structures from it. Also when I pass the pointer to these to another (forked)process that process should be able to access that memory. – lucent Feb 17 '12 at 15:24