0

I am building a python wrapper around a C++/CUDA project (called DEM-Engine) with the assistance of pybind11 and setuptools for packaging. My build works successfully and running pip3 install on the PyPi package DEME builds everything successfully from the sdist package.

However, my module DEME relies on a shared library that is built during the build process of the DEME package. To try to link my module to the shared library, I am adding the following to my Extension function definition:

conda_prefix = os.environ.get("CONDA_PREFIX")

super().__init__(name, sources=[], extra_link_args=['-Wl,-rpath,' + conda_prefix + '/lib/python3.10/site-packages/lib'], libraries=["libDEMERuntimeDataHelper"], include_dirs=["./src/DEM/", "./src/DEM/VariableTypes"], extra_objects=["./build"])

Via this method, I can ensure that the location of the shared library (which is always placed in the lib folder in site-packages) can be found. However, when importing my module I run into the error that the shared library could not be found.

Appending the location conda_prefix + '/lib/python3.10/site-packages/lib to my LD_LIBRARY_PATH ensures that everything is found and everything works correctly then.

Could someone please help me understand what I can do to ensure the end user does not have to define the LD_LIBRARY_PATH variable every time they install the package?

user3148282
  • 1
  • 1
  • You can use a tool described here https://stackoverflow.com/questions/13769141/can-i-change-rpath-in-an-already-compiled-binary or use `-rpath` with the linker. – 273K Aug 25 '23 at 00:28
  • I have tried to set runtime_library_dirs as well, however after checking the output of ldd for my module, I can see that the directories aren't searched. I was hoping to come up with a solution that would not require special work for the end user. Is there another way? – user3148282 Aug 25 '23 at 03:05
  • If rpath does not work for you, then you can use `dlopen()` and `dlsym()`. – 273K Aug 25 '23 at 07:06

0 Answers0