caffe can't find libboost_system.so.166.0

Question

I'm installing caffe on an Ubuntu 17.04 system using boost 1.66. I am able to execute make all and make test without problems:

me@icvr1:~/PackageDownloads/caffe$ make all
make: Nothing to be done for 'all'.
me@icvr1:~/PackageDownloads/caffe$ make test
make: Nothing to be done for 'test'.

However, when I try make runtest, I get the following error:

me@icvr1:~/PackageDownloads/caffe$ make runtest
.build_release/tools/caffe
.build_release/tools/caffe: error while loading shared libraries: libboost_system.so.1.66.0: cannot open shared object file: No such file or directory
Makefile:532: recipe for target 'runtest' failed
make: *** [runtest] Error 127

Now, I know that libboost_system.so.1.66.0 exists in /usr/local/lib, which (I believe) is a fairly standard location:

me@icvr1:~/PackageDownloads/caffe$ ls /usr/local/lib/libboost_system*
/usr/local/lib/libboost_system.a  /usr/local/lib/libboost_system.so  /usr/local/lib/libboost_system.so.1.66.0

and, in caffe's Makefile.config, /usr/local/lib is on the library path:

LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial

So, what am I missing here? How can I ensure caffe will know where to find libboost_system.so.1.66.0 ?

Answer 1

Your /usr/local/lib/libboost_system.so.1.66.0 is evidently one you built yourself and want the loader to find at runtime without special measures.

But having built it, you didn't update the ldconfig cache (because you didn't know you had to); so the runtime loader is not yet aware that this library exists and can't find it.

When the loader is looking for the shared libraries needed to assemble a new process, it does not search all of the linker's default search directories. That would be slow. By default it searches a cached database /etc/ld.so.cache, of libraries that were found by ldconfig, last time it was run.

By default, ldconfig caches libraries that it finds in /lib, /usr/lib and in the directrories listed in the file /etc/ld.so.conf, and/or any similar *.conf files that are recursively include-ed in /etc/ld.so.conf. For example:

$ cat /etc/ld.so.conf
include /etc/ld.so.conf.d/*.conf

$ cat /etc/ld.so.conf.d/*.conf
/usr/lib/x86_64-linux-gnu/libfakeroot
# libc default configuration
/usr/local/lib
# Multiarch support
/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu
/usr/lib/nvidia-384
/usr/lib32/nvidia-384
/usr/lib/nvidia-384
/usr/lib32/nvidia-384
# Legacy biarch compatibility support
/lib32
/usr/lib32

You see that /usr/local/lib is listed there. So, to make the loader aware of your new /usr/local/lib/libboost_system.so.1.66.0, just run:

ldconfig

as root. You need to do likewise anytime you install a new locally built shared library in /usr/local/lib.

If you want the loader to find a shared library /a/b/libfoo.so that isn't in the ldconfig cache, you can get it to do so by the special measure of appending /a/b/libfoo.so to the value of the environment variable LD_LIBRARY_PATH (which by default is empty) in the environment in which you launch the process that needs to load that library. The loader will search the LD_LIBRARY_PATH directories, if any, before the ldconfig cache. However, not adding a shared library to the ldconfig cache should be an informed choice and setting LD_LIBRARY_PATH is, of course, not well motivated merely by ignorance of the ldconfig apparatus or of the linker's -rpath option