Why can't this executable find this shared library?

Question

I am compiling an executable on linux through conan, meson and gcc. I am inspecting the final binary:

 readelf -d uproject | head -20
Dynamic section at offset 0xd4b00 contains 36 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libne_core.so]
 0x0000000000000001 (NEEDED)             Shared library: [libne_loader.so]
 0x0000000000000001 (NEEDED)             Shared library: [libcamera.so]
 0x0000000000000001 (NEEDED)             Shared library: [libgeometry.so]
 0x0000000000000001 (NEEDED)             Shared library: [libimage.so]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [/home/makogan/.conan/data/NeverEngine/0.0.1/_/_/package/fd44bedc483c60110edc71130be63a73237c36f4/lib]
 0x000000000000000c (INIT)               0x70000
 0x000000000000000d (FINI)               0xab9e4
 0x0000000000000019 (INIT_ARRAY)         0xd59a8
 0x000000000000001b (INIT_ARRAYSZ)       16 (bytes)
 0x000000000000001a (FINI_ARRAY)         0xd59b8
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x3b0

I then verify the runpath:

 ls /home/makogan/.conan/data/NeverEngine/0.0.1/_/_/package/fd44bedc483c60110edc71130be63a73237c36f4/lib
Animation.cpp.o                    HalfEdge_HMesh.cpp.o           libgeometry.so            libNeverEngine.so         Log.cpp.o                          Swapchain.cpp.o
AnimationMesh.cpp.o                HalfEdge_HMeshInternals.cpp.o  libgeometry.so.symbols    libprofiler.so            ModuleStorage.cpp.o                tracy_TracyClient.cpp.o
CacheUtils.cpp.o                   HalfEdge_Subdivision.cpp.o     libimage.so               libprofiler.so.symbols    NumericIntegration.cpp.o           Utils.cpp.o
Camera.cpp.o                       HardwareInterface.cpp.o        libimage.so.symbols       libspirv_cross_c.a        Parametrics_Parametrics.cpp.o      VkExtensionsStubs.cpp.o
CLI.cpp.o                          imgui_bridge.cpp.o             libimgui.so               libspirv_cross_core.a     Peripherals.cpp.o                  VulkanDebugging.cpp.o
Core.cpp.o                         _imgui.cpp.o                   libmisc_utils.so          libspirv_cross_cpp.a      Pipeline.cpp.o                     VulkanImage.cpp.o
CpuImage.cpp.o                     InputHandler.cpp.o             libmisc_utils.so.symbols  libspirv_cross_glsl.a     pkgconfig                          VulkanMemory.cpp.o
Gallery.cpp.o                      libanimation.so                libne_core.so             libspirv_cross_hlsl.a     profiler.cpp.o                     Window.cpp.o
GeometryUtils_GeometryUtils.cpp.o  libanimation.so.symbols        libne_core.so.symbols     libspirv_cross_msl.a      Renderer.cpp.o
GLFW_tools.cpp.o                   libcamera.so                   libne_loader.so           libspirv_cross_reflect.a  Scribe.cpp.o
GlslParsing.cpp.o                  libcamera.so.symbols           libne_loader.so.symbols   libspirv_cross_util.a     ShaderProgram.cpp.o
GltfLib.cpp.o                      libfilesystem.so               libne_peripherals.so      libui_utils.so            ShapeAnalysis_HeatGeodesics.cpp.o
GltfLoaders.cpp.o                  libfilesystem.so.symbols       libne_renderer.so         libui_utils.so.symbols    SpringMass.cpp.o

However when I run the program I get:

./uproject: error while loading shared libraries: libprofiler.so: cannot open shared object file: No such file or directory

Why? the files are clearly and exactly on the specified path. I see libprofiler.so among the files in the runpath directory.

Output with ldd:

linux-vdso.so.1 (0x00007fff4a6ff000)
        libne_core.so => /home/makogan/.conan/data/NeverEngine/0.0.1/_/_/package/fd44bedc483c60110edc71130be63a73237c36f4/lib/libne_core.so (0x00007f75f8a9c000)
        libne_loader.so => /home/makogan/.conan/data/NeverEngine/0.0.1/_/_/package/fd44bedc483c60110edc71130be63a73237c36f4/lib/libne_loader.so (0x00007f75f8a55000)
        libcamera.so => /home/makogan/.conan/data/NeverEngine/0.0.1/_/_/package/fd44bedc483c60110edc71130be63a73237c36f4/lib/libcamera.so (0x00007f75f896d000)
        libgeometry.so => /home/makogan/.conan/data/NeverEngine/0.0.1/_/_/package/fd44bedc483c60110edc71130be63a73237c36f4/lib/libgeometry.so (0x00007f75f84a5000)
        libimage.so => /home/makogan/.conan/data/NeverEngine/0.0.1/_/_/package/fd44bedc483c60110edc71130be63a73237c36f4/lib/libimage.so (0x00007f75f846e000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f75f823e000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f75f815a000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f75f8140000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f75f7f18000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f75f8c41000)
        libprofiler.so => not found
        libfilesystem.so => not found

Dump of libne_renderer.so:

Dynamic section at offset 0x68f750 contains 35 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libne_core.so]
 0x0000000000000001 (NEEDED)             Shared library: [libprofiler.so]
 0x0000000000000001 (NEEDED)             Shared library: [libfilesystem.so]
 0x0000000000000001 (NEEDED)             Shared library: [libX11.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libshaderc_shared.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libvulkan.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libne_renderer.so]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../Core:$ORIGIN/../Profiler:$ORIGIN/../../Tools/FileSystem]
 0x000000000000000c (INIT)               0x2a7000
 0x000000000000000d (FINI)               0x586f1c
 0x0000000000000019 (INIT_ARRAY)         0x68ec30
 0x000000000000001b (INIT_ARRAYSZ)       40 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x68ec58

And the ldd:

ldd lib/libne_renderer.so 
        linux-vdso.so.1 (0x00007ffddfbed000)
        libne_core.so => not found
        libprofiler.so => not found
        libfilesystem.so => not found
        libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007f362ca25000)
        libshaderc_shared.so.1 => /lib/x86_64-linux-gnu/libshaderc_shared.so.1 (0x00007f362c2de000)
        libvulkan.so.1 => /lib/x86_64-linux-gnu/libvulkan.so.1 (0x00007f362c26f000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f362c056000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f362bf72000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f362bf58000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f362bd30000)
        libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f362bd06000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f362bcff000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f362bcfa000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f362d24d000)
        libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007f362bcf4000)
        libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f362bcec000)
        libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f362bcd4000)
        libmd.so.0 => /lib/x86_64-linux-gnu/libmd.so.0 (0x00007f362bcc5000)

seems to not find 2 of the shared libraries, but don;t know why, they are both on the directory @ewong — Makogan, May 06 '22 at 07:56
check that these files are indeed shared libraries and that they have appropriate permissions. — n. m. could be an AI, May 06 '22 at 08:01
The two libraries are not in the list of libraries in the elf dynamic section that you included. I suspect that they are used by any of the other libraries, which not have the rpath set correctly. Could you try `ldd libne_core.so`, `ldd libne_loader.se` ,... and see which one of them uses `libprofiler.so` and/or `libfilesystem.so`. — Jakob Stark, May 06 '22 at 08:02
There's a few, one such file is the renderer one, i.e. libne_renderer.so — Makogan, May 06 '22 at 08:05
@JakobStark I added the outputs for one of the binaries (I fear doing all of them would make this post really long) — Makogan, May 06 '22 at 08:10
I am not 100% sure but I think, when a dynamic library is linked (during runtime) the dynamic linker tries to resolve the dependencies recursively. But for example for `libne_renderer.so`'s dependencies, only the rpath of that library is considered, not the original onn of the application. Thus it fails finding the right path. — Jakob Stark, May 06 '22 at 08:11
It's my packege (i.e. I am the author) but it was installed in the system by conan. @JakobStark — Makogan, May 06 '22 at 08:15
@Makogan Unfortunately I don't know the standard way of handling shared libraries in conan. But maybe [this](https://docs.conan.io/en/1.43/howtos/manage_shared_libraries.html) helps. — Jakob Stark, May 06 '22 at 08:31
Look at the runpath of libne_renderer.so. It looks like it expects to find each library (or a set of libraries) in its own subdirectory, presumably libne_core.so in Core/libne_core.so and libprofiler.so in Profiler/libprofiler.so etc, but instead all the libraries are lumped flat in one directory. It seems the configuration of the executable and the configuration of the libraries are not talking to each other. — n. m. could be an AI, May 06 '22 at 08:34
I think it is rather a problem in the interaction between the configuration of the library and the conan packaging procedure. The RPATH of the shared libraries should probably just be `$ORIGIN` and not `$ORIGIN/../Core:$ORIGIN/../Profiler:$ORIGIN/../../Tools/FileSystem`. — Jakob Stark, May 06 '22 at 08:43
@Makogan Maybe changing `keep_path` to `True` in [this](https://gitlab.com/Makogan/ne2/-/blob/master/conanfile.py#L125) line helps, because it reflects the directory structure of the build into the package layout. — Jakob Stark, May 06 '22 at 09:15

Why can't this executable find this shared library?

0 Answers0