4

Note that this is not a duplicate of Multiple instances of singleton across shared libraries on Linux since adding -rdynamic flag doesn't change what is described in the question.

I have a static c++ library which has some static initialization code (for variables requiring a constructor call in an unnammed namespace). I then have 2 (or more) shared library using that static library internally.

Any executable using both shared libraries will call the initialization code of the static library twice, making things like folly::Singleton or gflags global variables misbehave. Is that the intended behavior? Is that a bug/weakness of dynamic linkers?

edit: Note that on macos it only initialize once.

Example:

static.h

int get();

static.cpp

static int y = 0;
struct C { C() : x(y++) {}; int x; };
namespace { C c; }
int get() { return c.x; }

shared.h

int shared_get();

shared.cpp

#include "static.h"
int shared_get() { return get(); }

shared2.h

int shared2_get();

shared2.cpp

#include "static.h"
int shared2_get() { return get(); }

main.cpp

#include "shared.h"
#include "shared2.h"
#include <iostream>
int main() {
    std::cout << shared_get() << " " << shared2_get() << std::endl;
    return 0;
}

Compile:

g++ -fPIC -g -c  -o static.o static.cpp
ar rcs libstatic.a static.o
g++ -g -fPIC -shared -o libshared.so shared.cpp ./libstatic.a
g++ -g -fPIC -shared -o libshared2.so shared2.cpp ./libstatic.a
g++ main.cpp ./libshared.so ./libshared2.so

Run :

LD_LIBRARY_PATH=. ./a.out

The result is "1 1" when I would expect it to be "0 0".

Looking at the symbols with nm, libshared.so and libshared2.so do both contain:

t _GLOBAL__sub_I_static.cpp

Linking the static library only to the executable solves the behaviour but doesn't explain how two shared libraries can use a static library internally without interfering. For completion here is how to get "0 0":

Compile:

g++ -fPIC -g -c  -o static.o static.cpp
ar rcs libstatic.a static.o
g++ -g -fPIC -shared -o libshared.so shared.cpp
g++ -g -fPIC -shared -o libshared2.so shared2.cpp
g++ main.cpp ./libshared.so ./libshared2.so ./libstatic.a
ead
  • 32,758
  • 6
  • 90
  • 153
Léonard
  • 41
  • 6
  • Each shared library has **its own copy** of the static library. They don't know about each other. – Pete Becker Jul 06 '18 at 21:31
  • @PeteBecker Even with two copies, shouldn't each of them have their own `y` and `c`, and shouldn't both `c.x` be equal to 0? – Aconcagua Jul 06 '18 at 22:22
  • 1
    @Leonard: Please post an [mcve] (i. e. code of main as well). – Aconcagua Jul 06 '18 at 22:23
  • @Aconcagua — yes; I didn’t read the question carefully enough. – Pete Becker Jul 06 '18 at 23:06
  • @PeteBecker They don't know about each other until they are loaded at the same time: the symbol for `get` for example will not be loaded twice, only the first one loaded will be used, right? So I am kind of puzzled about why the call to the initialization is called twice too. – Léonard Jul 07 '18 at 23:16
  • @Aconcagua Thank you, I forgot to post it for some reason. It is added now. – Léonard Jul 07 '18 at 23:17
  • @PeteBecker Changing the static function to `int get() { static int x = 0; return x++; }` will convince you that after loading they are not 2 copies since both way of compiling will give the output "0 1". – Léonard Jul 07 '18 at 23:20
  • The robust solution is: with c++, don’t use static libraries for building shared libraries - use only shared libraries as dependencies of shared libraries. Machine code created by c++ (among others initialization of globals) assumes the One Definition rule to hold, otherwise there is undefined behavior. Violating ODR is not the end of the world, but adds complexity, that I would rather not have in my projects. – ead Jul 08 '18 at 04:25
  • I cannot test it now, but building static.o with -fvisibility=hidden should lead to “0 0” - i.e. each version of y and c being initialized once. But it is not what you want - they still would not be singletons. – ead Jul 08 '18 at 04:58
  • Sorry to say: not the linker/loader is at fault here but the coder.The rules are clear: C++ does depend on ODR and this is responsibility of the coder to make sure it isn’t violated. – ead Jul 08 '18 at 05:05
  • @ead well, It is not unreasonable to think that two independently build shared library could decide to internally use the same static library, no? Or it is a rule that shared library are never built with using 3rdparty static libraries? – Léonard Jul 08 '18 at 05:15
  • @ead I might be not understanding well, but ODR is kind of changed by dynamic loading (with the weak symbols, etc). Interestingly using `static int xx = xx++; int get() { return xx; }` shows my expected behavior on linux. (while both version do as expected on macos). – Léonard Jul 08 '18 at 05:24
  • To some degree weak symbols fix ODR violations. However, the way the initialization of globals is solved in c++ it just does work that well. As long as you only use Old plain data you might not see problems (as in your simplified example), but as soon as real classes/ constructors are used those initializer are called from `finit(`) function of different shared libraries, I.e. initialized multiple times. – ead Jul 08 '18 at 05:57
  • @ead Thank you for your answers. Things are quite dark in that corner of C++. Reading the ODR section of https://en.cppreference.com/w/cpp/language/definition It states two kind of contradicting things: "if the definition is for a class with an implicitly-declared constructor, every translation unit where it is odr-used must call the same constructor for the base and members" and "If all these requirements are satisfied, the program behaves as if there is only one definition in the entire program. Otherwise, the behavior is undefined." ... – Léonard Jul 08 '18 at 06:01
  • @EmployedRussian I don’t think it is a duplicate: they have in common that ODR is violated, but the setups are quite different and, at least to somebody inexperienced like me, there are a lot of gaps to be filled out from the given solution to understanding what is going on here. – ead Jul 08 '18 at 06:05
  • @Léonard this is the part, where C++ uses weak symbols to “fix” ODR “violations”. Those implicitly declared constructors are emitted as weak symbols in every translation unit (or inlined) and because all translations are equal, it doesn’t matter which one is picked by the linker – ead Jul 08 '18 at 06:14
  • @ead Well... Then `struct C { int x = y++; };` has implicit constructor and is still behaving like my first example... – Léonard Jul 08 '18 at 06:18
  • And every static library in my ubuntu under `/usr/lib` seems to be compiled with fPIC including boost, etc (according to tests found here https://stackoverflow.com/questions/1340402/how-can-i-tell-with-something-like-objdump-if-an-object-file-has-been-built-wi) – Léonard Jul 08 '18 at 06:29
  • Maybe I could reword and simplify my question to got other people understanding of ODR here. I am finding tons of potentially dangerous things by simply googling for PIC and static library... – Léonard Jul 08 '18 at 06:31
  • I understand. Too bad that `-fuse-ld=gold -Wl,--detect-odr-violations` doesn't detect it :/ – Léonard Jul 08 '18 at 06:44
  • I think this is an interesting and very complex topic/question and comments are not the best place to lead the discussion. My last remark: you probably can make it work somehow - but this adds complexity and somebody else working with the code must also understand all the subtle things + other systems (windows) will have different behavior. I personally keep it simple: no static libraries used to build shared objects - there is so much I don’t need to know now:) – ead Jul 08 '18 at 06:53

1 Answers1

3

To further simplify the problem you're getting results equivalent to having

static.cpp:

static int y = 0;
struct C { C() : x(y++) {}; int x; };
static C c; 
int get() { return c.x; }

and

shared1.cpp:

#include "static.h"
#include "static.cpp"
int shared1_get() { return get(); }

shared2.cpp:

#include "static.h"
#include "static.cpp"
int shared2_get() { return get(); }

(IOW, each of the 2 DSOs will include all of the static library's code, so we may just skip the static library part to make the example a little simpler.)

In your case

static C c; 

is essentially a per-DSO call to struct C's constructor, which in your case (without a hidden attribute on the struct) is exported.

Since you have two libraries with static C c;, a constructor for c will get called once in each DSO, and since the constructor symbol is exported (default), one of the constructors will win (along with its associated int get();) and will drag its statics with it.

Essentially, exactly one of the static C c; will be used and it will be constructed twice. That may sound weird but as far as shared libs are concerned, you should be thinking C, not C++, and see static C c; as

/*C*/
static int y;
static struct C c;
__attribute__((constructor))
void C__ctor(struct C *this) /*exported function*/
{
    C.x=y++;
}

To fix the problem you can mark the whole C struct as hidden so that that attribute applies to the constructor too

struct __attribute__((visibility("hidden")))  C { C() : x(y++) {}; int x; };

or better yet, compile with -fvisibility=hidden and apply visibility("default") attributes explicitly.

Below is an executable (shell script) example:

#!/bin/sh -eu
echo 'int get();' > static.h
cat > static.cpp <<EOF
static int y = 0;
#if 1 /*toggle to play with the visibility attribute*/
    #define MAYBE_HIDDEN __attribute__((visibility("hidden")))
#else
    #define MAYBE_HIDDEN
#endif
struct MAYBE_HIDDEN C { C() : x(y++) {}; int x; };
static C c; 
int get() { return c.x; }
EOF
cat > shared.h <<EOF
int shared_get();
EOF

cat > shared.cpp <<EOF
#include "static.h"
#include "static.cpp"
int shared_get() { return get(); }
EOF

cat > shared2.h <<EOF
int shared2_get();
EOF

cat > shared2.cpp <<EOF
#include "static.h"
#include "static.cpp"
int shared2_get() { return get(); }
EOF
cat > main.cpp <<EOF
#include "shared.h"
#include "shared2.h"
#include <iostream>
int main() {
    std::cout << shared_get() << " " << shared2_get() << std::endl;
    return 0;
}
EOF
g++ -fPIC -g -c  -o static.o static.cpp
#ar rcs libstatic.a static.o
g++ -g -fPIC -shared -o libshared.so shared.cpp #./libstatic.a
g++ -g -fPIC -shared -o libshared2.so shared2.cpp #./libstatic.a
g++ main.cpp ./libshared.so ./libshared2.so
./a.out

I've skipped the static lib, and included the C++ code directly, but you might as well revert that -- it doesn't change the results.

If you compile without the hidden attribute, you might want to try and run nm -D libshared.so libshared1.so. The _ZN1CC1Ev or the _ZN1CC2Ev symbol you should be getting there (or not, if you've applied the hidden attribute) should be the exported constructor function.

Petr Skocik
  • 58,047
  • 6
  • 95
  • 142
  • I do understant what is happening, I am mostly asking why it is happening. Why when loading the second library, the global symbol is skipped while its initialization (Constructor) code is not? Where in the standard or in linker can I find that behavior described? On macos the dynamic linker seems to be smarter since it only calls initialization once. – Léonard Jul 10 '18 at 18:59
  • 1
    @Léonard I'd have to consult the C++ standard (which is something I try to avoid so hard I'm using my own extended C) but my guess is it's probably an unintended bug of the Linux shared library implementation with respect to C++. The fact that macOS doesn't have the behavior would seem to confirm it. AFAIK, you're not violating any C++ edict, but the Linux implementation detail of constructor on the platform is inadvertently leaking. – Petr Skocik Jul 10 '18 at 19:16