Multiprocess c++(11) with linked list pointer as global variable

Question

I have the classic problem as given here, here and here and also here,

However, I would like a child process to insert an element at the end of a doubly linked list. a point to the first element of the list is global, and I want to access all of the list elements form the main process, and also next time I branch the main using fork, I want to be abe to access all elements, and update them, and insert more elements, in turn the main process again being able to access the modifyied list.

Each process exits with a system call with execvp (i need them to be able to call stuff using varied number of arameters).

I perhaps am asking a too broad question, but I personally did not get any further than branching and the inserting an element at the end of the list. Thus I actually dont have a single line of code that takes me where I want to go. I have no idea how to use shm() in this scenario.

Please help.

the threads will share the address space, so their is no need of shm() (make sure you know if you are making a new process or thread) and the rest depends on your exact situtaion - are they al reading or writing or both?, etc — user3125280, Jan 11 '14 at 13:43
Using shared memory is not necessarily enough -- especially if pointers in your shm point to memory that is not shared. It sounds though like you should not be using processes at all - rather you should be using threads (pthreads, or whatever your environment supports) within the same process. One reason is that the shared global space is only shared until a write occurs... then it's two separate memory buffers. — mah, Jan 11 '14 at 13:43
I used fork to branch carrying the use inputs, then based on these inputs, update the globals , then start an external process, then kill the child process, such that, before the next user input comes, the main process knowes what exactly was going on in the last call. I realize, a brutal slution would be to do the updating BEFORE the forking, or since the user inputs are also copied, let the child call the external program only, while the main updates the variables. But can I actually do the updatings in the child, instead of the brutal solution? — Sean, Jan 11 '14 at 13:50
If the child is a thread of the same process, and not a different process, then yes. If they're different processes, the child starts with a _copy_ of the global space, so writes by the child won't be seen by the parent. Since threads are in the same process, they don't have just a copy, they have the exact same buffer. (Implementation confusion: the fork child has read-only access to the original global space at first... on the first write, it gets a copy made, then read-write access. This is just for runtime efficiency, doesn't affect you.) With shm anyone can update but use synchronization! — mah, Jan 11 '14 at 13:56
does not fork() actually create a new process? So i guess after the forking I am having two seperate processes... And, well, I would really love an example of shm() with synchronization with linked lists. also do i need to pass each element of the list to shm? — Sean, Jan 11 '14 at 13:59

Brandon · Accepted Answer · 2014-01-11T15:46:48.897

You can try this.. I just wrote it from scratch.. It is cross-platform so that's always a plus. The allocators and pools can be re-used with anything. For example, you can make any stl container allocate on the stack or wherever you want..

SharedMemory.hpp:

#ifndef SHAREDMEMORY_HPP_INCLUDED
#define SHAREDMEMORY_HPP_INCLUDED

#if defined _WIN32 || defined _WIN64
    #include <windows.h>
#else
    #include <sys/types.h>
    #include <sys/mman.h>
    #include <dlfcn.h>
    #include <fcntl.h>
    #include <unistd.h>
#endif

#include <string>
#include <cstdint>

class SharedMemory
{
    private:
        std::string name;
        std::size_t size;
        void* data;
        void* hFileMap;

    public:
        SharedMemory(std::string name, std::size_t size) : name(name), size(size), data(nullptr), hFileMap(nullptr) {};
        ~SharedMemory();

        bool Open();
        bool Create();

        std::size_t GetSize() const {return this->size;}
        void* GetPointer() const {return this->data;}
};

#endif // SHAREDMEMORY_HPP_INCLUDED

SharedMemory.cpp:

#include "SharedMemory.hpp"

SharedMemory::~SharedMemory()
{
    if (data)
    {
        #if defined _WIN32 || defined _WIN64
        UnmapViewOfFile(data);
        data = nullptr;

        if (hFileMap)
        {
            if (CloseHandle(hFileMap))
            {
                hFileMap = nullptr;
            }
        }

        #else

        if (data)
        {
            munmap(data, size);
            data = nullptr;
        }

        if (hFileMap)
        {
            if (!close(hFileMap))
            {
                hFileMap = nullptr;
            }
        }
        #endif
    }
}

bool SharedMemory::Open()
{
    #if defined _WIN32 || defined _WIN64
        if ((hFileMap = OpenFileMapping(FILE_MAP_ALL_ACCESS, false, name.c_str())) == nullptr)
        {
            return false;
        }

        if ((data = MapViewOfFile(hFileMap, FILE_MAP_ALL_ACCESS, 0, 0, size)) == nullptr)
        {
            CloseHandle(hFileMap);
            return false;
        }
    #else

        if ((hFileMap = open(MapName.c_str(), O_RDWR | O_CREAT, 438)) == -1)
        {
            return false;
        }

        if ((data = mmap(nullptr, size, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, hFileMap, 0)) == MAP_FAILED)
        {
            close(hFileMap);
            return false;
        }
    #endif
    return true;
}

bool SharedMemory::Create()
{
    #if defined _WIN32 || defined _WIN64
        if ((hFileMap = CreateFileMapping(INVALID_HANDLE_VALUE, nullptr, PAGE_READWRITE, 0, size, name.c_str())) == nullptr)
        {
            return false;
        }

        if ((data = MapViewOfFile(hFileMap, FILE_MAP_ALL_ACCESS, 0, 0, size)) == nullptr)
        {
            CloseHandle(hFileMap);
            return false;
        }

    #else

        if ((hFileMap = open(MapName.c_str(), O_RDWR | O_CREAT, 438)) == -1)
        {
            return false;
        }

        if ((data = mmap(nullptr, Size, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, hFileMap, 0)) == MAP_FAILED)
        {
            close(hFileMap);
            return false;
        }
    #endif
    return true;
}

Pools.hpp:

#ifndef POOLS_HPP_INCLUDED
#define POOLS_HPP_INCLUDED

#include <stdexcept>
#include <cstdint>
#include "SharedMemory.hpp"

template<typename T>
class SharedPool
{
    private:
        T* data;
        SharedMemory* shm;
        std::size_t size;

    public:
        SharedPool(SharedMemory* shm) : data(reinterpret_cast<T*>(shm->GetPointer())), shm(shm), size(shm->GetSize()) {};

        template<typename U = T>
        void* allocate(std::size_t n, const void* hint = 0) {return &data[0];}

        template<typename U = T>
        void deallocate(U* ptr, std::size_t n) {}

        template<typename U = T>
        std::size_t max_size() const {return size;}
};

#endif // POOLS_HPP_INCLUDED

main.cpp (adding values to shared memory from process one):

#include "SharedMemory.hpp"
#include "Allocators.hpp"
#include "Pools.hpp"

#include <vector>
#include <iostream>

int main()
{
    SharedMemory mem = SharedMemory("Local\\Test_Shared_Memory", 1024);
    if (!mem.Open() && !mem.Create())
    {
        throw std::runtime_error("Error Mapping Shared Memory!");
    }

    auto pool = PoolAllocator<int, SharedPool<int>>(SharedPool<int>(&mem));
    std::vector<int, decltype(pool)> v(pool);

    int* ptr = reinterpret_cast<int*>(mem.GetPointer());
    std::cout<<"Pushing 3 values to: "<<ptr<<"\n";

    v.push_back(100);
    v.push_back(200);
    v.push_back(700);

    std::cin.get();
}

main.cpp (Reading values from shared memory process two):

#include "SharedMemory.hpp"
#include "Allocators.hpp"
#include "Pools.hpp"

#include <vector>
#include <iostream>

int main()
{
    SharedMemory mem = SharedMemory("Local\\Test_Shared_Memory", 1024);
    if (!mem.Open() && !mem.Create())
    {
        throw std::runtime_error("Error Mapping Shared Memory!");
    }

    auto pool = PoolAllocator<int, SharedPool<int>>(SharedPool<int>(&mem));
    std::vector<int, decltype(pool)> v(pool);

    int* ptr = reinterpret_cast<int*>(mem.GetPointer());
    std::cout<<"Reading 3 values from: "<<ptr<<"\n";

    v.reserve(3);
    std::cout<<v[0]<<"\n";
    std::cout<<v[1]<<"\n";
    std::cout<<v[2]<<"\n";

    std::cin.get();
}

success, this worked, thank you.but i might ask some more questions in later hours.. — Sean, Jan 11 '14 at 23:43
I don't mind. Ask away. I don't comment my code a lot so I hope you understood it. If not, feel free to ask anything you like. Myself or someone else (whoever sees it) will help. — Brandon, Jan 11 '14 at 23:46
for now, yes i do see how it is working. but you have to keep in mind that i am a weatherman with a mathematics background (trying to automate my weather related stuff myself - it is often easier do it yourself than to explain what exactly is needed for sake of usability) - it might be the case that i understood it wrong. Let's see . — Sean, Jan 11 '14 at 23:56

user1708860 · Answer 2 · 2014-01-11T17:49:30.730

1

This is a tough problem.

One way is to use Shared Memory and when you build the linked list, give it your own allocator, to use the shared memory. Other way is to implement your own linked list based on the shared memory.

You can also try to use boost - Boost interprocess, which is probably the ideal solution.

Specifically speaking - interprocess with containers

edited Jan 11 '14 at 17:49

answered Jan 11 '14 at 13:45

user1708860

1,683
13
32

The OP stated _I have no idea how to use shm() in this scenario_ so simply suggesting he use shared memory seems insufficient. – mah Jan 11 '14 at 13:50

Multiprocess c++(11) with linked list pointer as global variable

2 Answers2