Static vs. dynamic linking conflicts and duplication

Question

I have a code A that is statically linked against one version of mpich. Now comes library B, which is used by A via dlopen(). B depends on mpich as well, but is linked dynamically against it.

The problem is that now, in order for B to take advantage of mpi distribution, needs to access the communicator currently handled by A. This communicator has been created by A static version of mpich, When B invokes MPI routines, it will use a dynamic version of MPI which is not guarateed to be compatible with the static version attached to A.

This is the overall picture. I think that the only solution is to have mpich dynamically linked for both A and B. What I am not fully understanding is however the following:

• how does the linker handle shared objects dependencies when dlopening? Will I have two instances of mpich in VM also with dynamic linking, or is the linker smart enough to realize that the symbols required by the dlopened B are already in the address space and will resolve against those.
• Is it possible to tell the linker: when you dlopen this library, don't go fetch the dynamic dependency, but resolve it with the static symbols that are already provided by A

Answer 1

In short: it depends on dlopen options. By default, if a symbol needed by the requested library already exists in the global scope, it will be reused (this is what you want). But you can bypass this behavior with RTLD_DEEPBIND, with this flag, the dependencies won't be reused from the global scope, and will be loaded a second time.

Here is some code to reproduce your situation and demo the effect of this flag.

Let's make a common library that will be used by both lib A and program B. This library will exist in two versions.

$ cat libcommon_v1.c 
int common_func(int a)
{
    return a+1;
}
$ cat libcommon_v2.c 
int common_func(int a)
{
    return a+2;
}

Now let's write lib A that uses libcommon_v2:

$ cat liba.c 
int common_func(int a);

int a_func(int a)
{
    return common_func(a)+1;
}

And finally program B that dynamically links to libcommon_v1 and dlopens lib A:

$ cat progb.c
#include <stdio.h>
#include <dlfcn.h>

int common_func(int a);
int a_func(int a);

int main(int argc, char *argv[])
{
    void *dl_handle;
    int (*a_ptr)(int);
    char c;

    /* just make sure common_func is registered in our global scope */
    common_func(42);

    printf("press 1 for global scope lookup, 2 for deep bind\n");
    c = getchar();
    if(c == '1')
    {
        dl_handle = dlopen("./liba.so", RTLD_NOW);
    }
    else if(c == '2')
    {
        dl_handle = dlopen("./liba.so", RTLD_NOW | RTLD_DEEPBIND);
    }
    else
    {
        printf("wrong choice\n");
        return 1;
    }
    if( ! dl_handle)
    {
        printf("dlopen failed: %s\n", dlerror());
        return 2;
    }
    a_ptr = dlsym(dl_handle, "a_func");
    if( ! a_ptr)
    {
        printf("dlsym failed: %s\n", dlerror());
        return 3;
    }

    printf("calling a_func(42): %d\n", (*a_ptr)(42));

    return 0;
}

Let's build and run all the things:

$ export LD_LIBRARY_PATH=.
$ gcc -o libcommon_v1.so -fPIC -shared libcommon_v1.c
$ gcc -o libcommon_v2.so -fPIC -shared libcommon_v2.c
$ gcc -Wall -g -o progb progb.c -L. -lcommon_v1 -ldl
$ gcc -o liba.so -fPIC -shared liba.c -L. -lcommon_v2
$ ./progb 
press 1 for global scope lookup, 2 for deep bind
1
calling a_func(42): 44
$ ./progb 
press 1 for global scope lookup, 2 for deep bind
2
calling a_func(42): 45

We can clearly see that with default options, dlopen reuses the symbol common_func that was present in program B and that with RTLD_DEEPBIND, libcommon was loaded again and library A got its own version of common_func.

Answer 2

You did not say which Toolchain (GCC, LLVM, MSC, etc.) you are using, the most helpful answer will depend upon this information.

May I suggest you look at "GCC Exception Frames" http://www.airs.com/blog/archives/166 .

If that is helpful then the Gold Linker, which is available for GCC and LLVM, supports 'Link Time Optimization' and can run in "Make" with DLLTool http://sourceware.org/binutils/docs/binutils/dlltool.html .

Indeed it is possible to have both Static and Dynamic Code call each other, the Computer does not care; it will 'run' anything it is fed -- whether that ends up working exactly the way you want or HCFs depends on correct Code and correct Linker commands.

Using a Debugger will not be fun. It would be best to mangle the names prior to Linking so that when debugging you can see from which module the Code came. Once it is up and running you can undef the Mangle and have the same-named Functions Link (to ensure it still functions).

Compiler / Linker Bugs will not be your friend.

This sort of scenario (Static and Dynamic Linking) occurs more often with MinGW and Cygwin where some of the Libraries are Static yet a Library you download from the Internet is only available in Dynamic form (without Source).

If the Library is from two different Compiler Toolchains then other issues arise, see this StackOverflow Thread: " linking dilemma (undefined reference) between MinGW and MSVC. MinGW fails MSVC works ".

It would be best to simply get the newest version of the Library from the source and compile the whole thing yourself, rather than rely on trying to cobble together bits and pieces from different sources (though it is possible to do that).

You can even load the Dynamic Library and call it (statically) and then reload portions of it later.

How tight are you on Memory and how fast do you want Functions to run, if everything is in Memory your Program can transfer execution to called Functions straight away, if you are swapping a portion of your Code to VM your execution times will really take a hit.

Running a Profiler on your Code will help decide what portions of a Library to load if you want to do 'dynamic dynamic linking' (full control of your dyna-linking by loading a Dynamic Library so it can be used Statically). This is the stuff that headaches and nightmare are made of. GL.