Using custom glibc with bazel

Question

I am in a non-root (debian) environment and when compiling a binary I got a

Failed to open library! - ./libdmlab.so
dlopen: cannot load any more object with static TLS

I learned that this is a somewhat common problem and was circumvented in glibc/2.21.1 by increasing the limit. I am now trying to locally install a newer (2.22) version of glibc and link using it in bazel.

I have compiled glibc from the sources and added appropriate things to PATH, CPATH and LIBRARY_PATH. Adding glibc/lib to LD_LIBRARY_PATH made ldd segfault, so I removed it.

Now comes the question of using the locally installed glibc in bazel. I have tried using -linkeropt (as well as -copt) as:

bazel run --linkopt "-Wl,--rpath=/u/sygnowsj/bin/glibc -Wl,--dynamic-linker=/u/sygnowsj/bin/glibc/lib/ld-linux-x86-64.so.2" run //target

as described in this answer, but it didn't change anything. I also tried adding

"-Wl,--rpath=/u/sygnowsj/bin/glibc",
"-Wl,--dynamic-linker=/u/sygnowsj/bin/glibc/lib/ld-linux-x86-64.so.2",

to the linkeropts of the problematic library in the BUILD file, but it didn't help either.

How can I make bazel link against my version of glibc for all of the targets? Can I in any other way circumvent the problem of hitting static TLS limit?

Answer 1

bazel run --linkopt "-Wl,--rpath=/u/sygnowsj/bin/glibc -Wl,--dynamic-linker=/u/sygnowsj/bin/glibc/lib/ld-linux-x86-64.so.2" run //target

This doesn't look right: you have two runs in there. Also, passing a single -Wl,--rpath...ld-linux-x86-64.so.2 argument to the linker is not going to do what you want. You need either:

--linkopt=-Wl,--rpath=/u/sygnowsj/bin/glibc \
--linkopt=-Wl,--dynamic-linker=/u/sygnowsj/bin/glibc/lib/ld-linux-x86-64.so.2

or:

 --linkopt=-Wl,--rpath=/u/sygnowsj/bin/glibc,--dynamic-linker=/u/sygnowsj/bin/glibc/lib/ld-linux-x86-64.so.2

Finally, your --rpath value: /u/sygnowsj/bin/glibc looks wrong. It should likely be:

-Wl,--rpath=/u/sygnowsj/bin/glibc/lib

(since that's where your ld-linux resides).