4

I'm trying to load a shared library in C twice:

lib1 = dlopen("mylib.so", RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND);
lib2 = dlopen("mylib.so", RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND);

What I want is that lib1 and lib2 have separate address spaces so that they can do different things. Currently, the only way I can achieve this is by copying mylib so that the code looks like this:

lib1 = dlopen("mylib.so", RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND);
lib2 = dlopen("mylib2.so", RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND);

In a limited scope this works fine for me. However, I have an application which uses the library a generic number of times which makes copying the library cumbersome.

Is there a better way to have a separate address space for each time the library is loaded?

EDIT:

I want to load the library multiple times as my application is processing a kind of message queue. The items in the message queue refer to the name of a shared library (e.g. mylib) and contain a set of data that shall be processed by the library. I want to process the MQ in a multithreading environment, running each call to the library's method in its own thread. As long as the MQ contains the call to a library only once, everything is working as expected. However, when I have two items that use the same library, things start to get weird.

Mr Fooz
  • 109,094
  • 6
  • 73
  • 101
MiH
  • 115
  • 2
  • 13
  • 4
    So you want to shared library to *not* work like a shared library ;-) Why? – P.P Feb 19 '18 at 11:03
  • 1
    Please **edit your question** to improve it and explain *why you want to `dlopen` the same library twice*. Smells badly as some [XY problem](http://xyproblem.info/) – Basile Starynkevitch Feb 19 '18 at 11:31
  • 1
    @BasileStarynkevitch You are right, I'm sorry about this. I edited my question and I hope I added the necessary information – MiH Feb 20 '18 at 13:49
  • 1
    Concerning your description of why you want to do that, it seems that the library is not reentrant then. This happens when e.g. there is mutable state that is shared between different calls. However, I wouldn't rule out that your way of calling the code is at fault, although it is less likely since you are able to work around it by copying the library. – Ulrich Eckhardt Feb 20 '18 at 20:48

3 Answers3

7

You need to use dlmopen to achieve this sort of isolation:

// No need for RTLD_LOCAL, not sure about RTLD_DEEPBIND
lib1 = dlmopen (LM_ID_NEWLM, "mylib.so", RTLD_LAZY | RTLD_DEEPBIND);
eyllanesc
  • 235,170
  • 19
  • 170
  • 241
yugr
  • 19,769
  • 3
  • 51
  • 96
3

The whole idea of dynamically loading code is that you are thus able to share it, in particular with other processes. For that reason, I don't think that it is possible to really load the library twice.

There are ways around this though. One may be to fool the dynamic linker into loading it a second time. Copying the library is one way that you found already. I could imagine hard links to work, too.

However, I think it would be better if you worked with the flow here. I see two ways to achieve what I guess is your goal: Forking a separate process or creating a separate init function.

For the separate process, you just fork(), after setting up appropriate IPC mechanisms between parent and child instead of loading the library a second time. Since the fork creates a new process, it receives its own memory space and things remain separate. As IPC I'd suggest using some kind of middleware, like ZeroMQ, dbus or XMLRPC.

Creating a separate init function is the other option. For that, instead of creating the library's state as globals, you throw them together into a structure. Then, in that init function, you create an instance of that structure, set it up and returns its address. All other functions, which previously operated on the global state, now receive the address of that structure as additional (customary first) parameter. Instead of loading the library twice, you simple call the init function twice to set up separate environments.

Ulrich Eckhardt
  • 16,572
  • 3
  • 28
  • 55
  • I'm currently using pthreads to create the threads that each dlopen() is running in. As far as I understand the difference between fork() and pthreads is that fork creates a new address space and therefore requires a IPC middleware. Is there no way of direct communication between the processes? – MiH Feb 20 '18 at 14:28
  • 1
    "I don't think that it is possible to really load the library twice" - it is, using [dlmopen](http://man7.org/linux/man-pages/man3/dlopen.3.html). – yugr Feb 20 '18 at 15:23
  • If `dlmopen` works, it seems to be a suitable, non-hackish approach to loading said library twice. – Ulrich Eckhardt Feb 20 '18 at 20:41
-1

Is there a better way to have a separate address space for each time the library is loaded?

Actually, a virtual address space belongs to a process (so to all threads inside it), not to a shared library (which uses several segments of that virtual address space).

For a process of pid 1234, use pmap(1) (as pmap 1234) or proc(5) (e.g. try cat /proc/1234/maps ...)

You really should avoid dlopen(3)-ing the same shared library "twice" (and this is difficult, on purpose; you could use symlinks and dlopen several symlinks to the same shared object, but you should not do this, for example because static data would be "loaded twice" and aftermath will happen). To avoid this happening, the dynamic loader uses reference counting techniques...

Read also Drepper's How to Write Shared Libraries

Is there a better way to have a separate address space for each time the library is loaded?

You then need different processes, each having its own virtual address space. You'll use inter-process communication : see pipe(7), fifo(7), socket(7), unix(7), shm_overview(7), sem_overview(7) etc...

Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547
  • "You really should avoid dlopen(3)-ing the same shared library twice" - why? [dlmopen manpage](http://man7.org/linux/man-pages/man3/dlopen.3.html) mentions this use-case... – yugr Feb 20 '18 at 15:24