Is it possible to allocate a list into shared memory in C++?

Question

I am creating a server, where I have a class that handles the clients, a class that handles the server, and another class that displays each new client into a listbox. The way that I want to do it and have mostly working is, that I have the server running on a child process using fork, and the listbox running on the parent process. I want to be able to have a shared list that holds the different client classes. In python it would be done with multiprocessing manager. I have been unable to find something like that. The way I am currently attempting to do it is with mmap. This is basically the way I have made the code so far:

my relevant header files:

#include <list>
#include <sys/mman.h>

//creating the list
//the name of the client class is called ClientHolder
size_t pagesize = getpagesize();
std::list<ClientHolder> *Clients = static_cast<std::list<ClientHolder>*>(mmap(NULL, (pagesize*(1 << 20)),
                PROT_READ | PROT_WRITE,
                MAP_SHARED | MAP_ANONYMOUS, 0, 0));
//then it is passed into the server class as such:
Server newServer(*Clients);
//then initialized in my listbox class as well:
ListboxHandler newListboxHandler(listbox, *Clients);

This is what my Server class initialization looks like:

class Server{
    bool running=false;
    char *ip;
    int Port;
    std::list<ClientHolder> *Clients;
    size_t pagesize = getpagesize();
    int *quit_call = static_cast<int*>(mmap(NULL, (pagesize*(1 << 20)),
                PROT_READ | PROT_WRITE,
                MAP_SHARED | MAP_ANONYMOUS, 0, 0));
    public:
        Server(std::list<ClientHolder> Clients){Clients = Clients;};

Also, the quit_call variable works as I want it to. Obviously there is a lot more to my classes and my code. I just wanted to show exactly what was relevant.

It should also be noted that this does compile without any warnings or errors using clang++. Except that every time that I run this I get a zsh: segmentation fault which means it is trying to read and or write to illegal memory. Is there a way to do what I want?

The little code you've shown us can't be real code. The constructor has two variables both called `Clients` and looks like it's trying to assign a `std::list` to a variable of type `std::list*`. Also, both variables are called `Clients`, so `Clients = Clients;` doesn't do anything. — David Schwartz, Dec 01 '20 at 22:47
static_cast to `std::list` does not seem safe. That's an artifact of C, but is not a good approach in C++ — smac89, Dec 01 '20 at 22:47
You're using the default allocator for std::list. It isn't going to allocate things it needs out of your shared memory section and even if it did you'd still be in trouble since the shared memory pointer would likely start at a different virtual address in each process, and that's not even addressing locking. You might want to look at this answer: https://stackoverflow.com/questions/5363602/is-it-possible-to-place-stdvector-to-shared-memory — George, Dec 01 '20 at 22:47
https://www.boost.org/doc/libs/1_51_0/doc/html/interprocess/quick_guide.html#interprocess.quick_guide.qg_interprocess_container Boost.Interprocess allows creating complex objects in shared memory and memory mapped files. For example, we can construct STL-like containers in shared memory. To do this, we just need to create a special (managed) shared memory segment, declare a Boost.Interprocess allocator and construct the vector in shared memory just if it was any other object. — Mooing Duck, Dec 01 '20 at 23:10
Rather than shared memory, why not go for observer pattern. When a client connects, your server sends a notification to the main application containing information about the client (could just be an ID). You can use IPC or any form of TCP to communicate this information. If you really like shared memory, another option is to use a key-value store like reddis to store the client info, and have your main application subscribe to the database to retrieve new clients — smac89, Dec 01 '20 at 23:13

score 2 · Answer 1 · answered Dec 01 '20 at 22:48

std::list<ClientHolder> *Clients = static_cast<std::list<ClientHolder>*>(mmap(NULL, (pagesize*(1 << 20)),
            PROT_READ | PROT_WRITE,
            MAP_SHARED | MAP_ANONYMOUS, 0, 0));

This won't work. The std::list class allocates memory and stores pointers. Those pointers will not be valid in another process. You cannot share instances of std::list that way because each process will write pointers to its own memory space into the instances of std::list in shared memory and the other process will crash when it tries to dereference them.

You will need to allocate a lot more shared memory, enough to hold everything you want to share. And you'll need to give std::list an allocator that only assigns chunks of that shared memory so that the other process will understand the pointers placed into shared memory.

This is probably more trouble than it's worth and you probably would do better to reconsider the design.

In the code posted, it looks like the list is being copied (`Server newServer(*Clients);`), so won't this bypass the problem you highlighted? As you can see, `Server` takes `Clients` by value — smac89, Dec 01 '20 at 22:52
@smac89 No, it won't. When the list is copied, new memory will be allocated for the copy, and it won't come out of shared memory because the list is just constructed with the default allocator which calls `new` which probably calls `malloc`. He has to modify the `std::list` allocator to allocate from shared memory, otherwise the objects will contain pointers to unshared memory. — David Schwartz, Dec 01 '20 at 22:58

score 1 · Answer 2 · answered Dec 01 '20 at 22:58

1

I agree with @DavidSchwartz that this is a bad design, but for another reason.

You can take care of segmentation fault by using an array instead of the list, and by ensuring that your ClientHolder is a flat data structure (without external pointers).

But how are you going to update that array? Like add / remove elements? You would need some kind of synchronization that might negate the benefits of shared memory.

A question: why your Server can't handle that listbox?

answered Dec 01 '20 at 22:58

Vlad Feinstein

10,960
1
12
27

I disagree that synchronization negates the benefits of shared memory. – Mooing Duck Dec 01 '20 at 23:05
@MooingDuck - there is nothing to agree or disagree - we don't know what is the nature of that shared data, and how that data is used. The main point was: the `Server` can handle that listbox, without any child (or parent?) process and IPC – Vlad Feinstein Dec 01 '20 at 23:20

Is it possible to allocate a list into shared memory in C++?

2 Answers2