Why do I need to set LD_LIBRARY_PATH after installing my binary?

Question

My application is linking to a self-compiled Qt that lies in /usr/local/lib.

This works fine when I start my application in the build dir. However, after I have installed my application to /usr/local/bin/ it tries to load incompatible system Qt lib that doesn't have required symbols and fails.

Why? How does the actual location of my binary affect which Qt is being resolved?

If I start my app with LD_LIBRARY_PATH=/usr/local/lib it works like intended.

Check https://unix.stackexchange.com/questions/67781/use-shared-libraries-in-usr-local-lib or https://stackoverflow.com/questions/4743233/is-usr-local-lib-searched-for-shared-libraries some distribution choose not include `/usr/local/lib` in library search path — Zang MingJie, Aug 05 '19 at 12:13
can you check the rpath of the binary ? `objdump -x BINARY | grep RPATH`, maybe there is `.` inside RPATH. `RPATH` works similar as `LD_LIBRARY_PATH`, but defined inside binarys — Zang MingJie, Aug 05 '19 at 12:18
try `LD_DEBUG=all prog`, it usually gives enough information to diagnose the problem. — Zang MingJie, Aug 05 '19 at 12:24

jww · Accepted Answer · 2019-08-05T23:48:25.260

Why do I need to set LD_LIBRARY_PATH after installing my binary?

The simple answer is, it is a failure of the system architects and the toolchain. On Linux they are the folks who maintain Binutils and glibc. Collectively the maintainers think it is OK to compile and link against one version of a library, and then runtime link against the wrong version of the library or lose the library. They have determined it is the number one use case (q.v.). Things don't work out of the box, and you have to do something special to get into a good state.

Consider, I have some Build Scripts that build and install about 70 common Linux utilities and libraries, like cURL, Git, SSH and Wget. The scripts build the program and all of its dependencies. Sometimes I attempt to use them for security testing and evaluation, like Asan and UBsan. The idea is to run the instrumented program and libraries in /usr/local and detect undefined behavior, memory errors, etc. Then, report the error to the project.

Trying to run instrumented builds on a Linux system is impossible. For example, once Bzip, iConv, Unicode, IDN, PRCE, and several other key libraries are built with instrumentation the system no longer functions. Programs in /usr/bin, like /usr/bin/bash, will load instrumented libraries in /usr/local/lib. The programs and libraries have so many faults the programs are constantly killed. You can no longer run a script because the programs in /usr/bin are using wrong libraries (and programs like bash have so many faults).

And of course there are problems like yours, where programs and libraries can't find its own libraries. The web is full of the carnage due to the maintainers' decisions.

To fix the issue for the program you are building, add the following to your LDFLAGS:

-Wl,-R,/usr/local/lib -Wl,--enable-new-dtags

-R,/usr/local/lib tells the runtime linker to look in /usr/local/lib for its libraries. --enable-new-dtags tells the linker to use RUNPATH as opposed to a RPATH.

RUNPATH's are overrideable with LD_LIBRARY_PATH, while RPATH's are not. Omitting -Wl,--enable-new-dtags is usually a bad idea. Also see Use RPATH but not RUNPATH?.

You can also use a relative runtime path:

-Wl,-R,'$$ORIGIN/../lib' -Wl,--enable-new-dtags

-R,'$$ORIGIN/../lib' tells the runtime linker to look in ../lib/ for its libraries. Presumably your binary is located in .../bin/, so ../lib/ is up-and-over to the the lib/ directory.

ORIGIN-based linker paths are good when you have a filesystem layout like:

My_App
  |
  +-- bin
  |    |
  |    +- my_app.exe
  |
  +-- lib
  |    |
  |    +- my_lib.so
  |
  +-- share

You can move My_App folder around the filesystem, and my_app.exe will always be able to find its library my_lib.so. And if the My_App is the root directory /, then it is a standard Linux file system used by distros (i.e., --prefix=/). And if the My_App is /usr/local, then it is a standard Linux file system used by the user (i.e., --prefix=/usr/local).

If you configure, make and make test, then you can use LD_LIBRARY_PATH to set the temporary path to the library for testing (like .../My_App/lib). In fact, a well written make test should do that for you during testing.

You should not have to add -Wl,-R,/usr/local/lib -Wl,--enable-new-dtags since the program was compiled and linked against the library in /usr/local/lib (presumably using -L<path> and -l<lib>). The reason you have to do it is because the maintainers keep making the same mistake over and over again. It is a complete engineering failure.

The Linux architects were warned about these path problems about 25 years ago when they were deciding what to do out of the box. There were two camps - one who wanted things to work out of the box, and one who wanted users to do something special to get things to work. The later won the debate. They chose the "let's link against the wrong library" and "let's lose the library" patterns. I wish I could find the Usenet posting where it was discussed so you could read it for yourself.

This is a very useful write up! Thank you. Occasionally I would have to run utility which is not compatible with the current libs. This would allow me to run that with its own libs. For understanding the basics like this is very helpful. — CqN, Mar 19 '22 at 21:36

Why do I need to set LD_LIBRARY_PATH after installing my binary?

1 Answers1