3

I am working on a package which includes C code from third-party library (SUNDIALS). The package compiles and works (i.e., is able to solve a test ODE) with the following Makevars file performing static linking

CXX=clang++

PKG_CPPFLAGS = -I../inst/include
PKG_LDFLAGS = /usr/local/lib
PKG_LIBS= $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) $(PKG_LDFLAGS)/libsundials_cvode.a $(PKG_LDFLAGS)/libsundials_nvecserial.a

However, a slightly modified version (based on the example in R-Exts, i.e. - PKG_LIBS = -L$(XML_DIR)/lib -lxml2) of Makevars (below) fails

CXX=clang++

PKG_CPPFLAGS = -I../inst/include
PKG_LDFLAGS = /usr/local/lib
PKG_LIBS= $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) -L$(PKG_LDFLAGS) -lsundials_cvode -lsundials_nvecserial -lm

fails with the following error message.

Error: package or namespace load failed for ‘Rcppsbmod’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod/libs/Rcppsbmod.so':
  dlopen(/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod/libs/Rcppsbmod.so, 6): Library not loaded: libsundials_cvode.3.dylib
  Referenced from: /Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod/libs/Rcppsbmod.so
  Reason: image not found
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod’

Exited with status 1.

I am not sure why it is looking for the libraries in another location when I am specifying PKG_LDFLAGS as /usr/local/lib.

As an aside, the test example which comes which the SUNDIALS package compiles and works with the following command

gcc -Wall cvRoberts_dns.c -o cvRoberts_dns.exe -I/usr/local/include -L/usr/local/lib/ -lsundials_cvode -lsundials_nvecserial -lm

So, I know that the library is installed properly and correct files (for linking) are available at /usr/local/lib location.

The entire package source code can be found at - https://github.com/sn248/Rcppsbmod

Any help or guidance will be highly appreciated!

Satya
  • 1,708
  • 1
  • 15
  • 39

2 Answers2

3

System-wide dynamic linking, as in your second use case which fails, requires the cooperation of the dynamic linker on your system.

That means after build and copyring the libraries to /usr/local/lib you must typically run sudo ldconfig to update the linker cache.

You can check if the libraries are know by grep'ing through the output of ldconfig -p. On my system, no sundials:

edd@rob:~$ ldconfig -p | grep sundials
edd@rob:~$

Relatedly you can (locally) use different directories by declaring them in /etc/ld.so.conf.d/somefile.conf -- but that is of course not portable and would not help you with a package designated for CRAN.

The use of static libraries you build as part of your package as in your first example would work as it does not require any system assistance. It just takes longer the build the libraries each time.

Dirk Eddelbuettel
  • 360,940
  • 56
  • 644
  • 725
  • Thank you @Dirk for your suggestion. On OSX (should have mentioned that before) there is no `ldconfig`, but there is `sudo update_dyld_shared_cache`. Running this command `sudo update_dyld_shared_cache | grep sundials` gives a long message such as `update_dyld_shared_cache: warning @rpath install name for system framework: /System/Library/PrivateFrameworks/AssetCacheServicesExtensions.framework/Versions/A/XPCServices/AssetCacheTetheratorService.xpc/Contents/Frameworks/MobileDeviceKitLite.framework/Versions/A/MobileDeviceKitLite`. – Satya Mar 07 '18 at 20:28
  • However, if static vs dynamic is not that important, I'll not make any changes, for now. My next challenge is to bundle the package such that there is no pre-requisite of having the library installed on the system (as installing sundials on Windows is not that trivial). Thanks for your answer though. – Satya Mar 07 '18 at 20:32
2

I am fighting with similar issues, c.f. Runtime linking R-extension on MacOS. My current workaround is to set the rpath at compile time. In your case that would mean:

CXX=clang++

PKG_CPPFLAGS = -I../inst/include
PKG_LDFLAGS = /usr/local/lib
PKG_LIBS= $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) -L$(PKG_LDFLAGS) -lsundials_cvode -lsundials_nvecserial -lm -Wl,-rpath,$(PKG_LDFLAGS)

However, this does not fix your problems. Comparing the error messages I see one difference: In your case the library libsundials_cvode.3.dylib is not found, while in my case it is @rpath/libaf.3.dylib. This means that the library you installed identifies itself as libsundials_cvode.3.dylib. You can check this with 

$ otool -L /usr/local/lib/libsundials_cvode.3.dylib 
/usr/local/lib/libsundials_cvode.3.dylib:
    /usr/local/opt/sundials/lib/libsundials_cvode.3.dylib (compatibility version 3.0.0, current version 3.1.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)

In your case the second output line should not contain an absolute path but only the basename of the library. My installation uses brew, which typically uses absolute paths as library names. In some trivial tests I had no problem with linking an R extension with these libraries.

I see several possibilities:

  • Try SUNDIAL from brew.

  • Adjust the library path in your installed libraries with

    install_name_tool -id /usr/local/lib/libsundials_cvode.3.dylib /usr/local/lib/libsundials_cvode.3.dylib

    to use absolute paths.

  • Adjust the library path in your installed libraries with

    install_name_tool -id '@rpath/libsundials_cvode.3.dylib' /usr/local/lib/libsundials_cvode.3.dylib

    and set rpath as above.

  • Adjust the name of the library your R extension is looking for with this addition to Makevars

    all: $(SHLIB)
    @if command -v install_name_tool; then install_name_tool -change libsundials_cvode.3.dylib /usr/local/lib/libsundials_cvode.3.dylib $(SHLIB); fi
Ralf Stubner
  • 26,263
  • 3
  • 40
  • 75
  • Ralf - Thanks for your answer. I tried your approach (slight change - with `-rpath` instead of `rpath`). I am getting following message (see below). Any suggestion is most welcome! – Satya Mar 08 '18 at 23:55
  • `Error: package or namespace load failed for ‘Rcppsbmod’ in dyn.load(file, DLLpath = DLLpath, ...): unable to load shared object '/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod/libs/Rcppsbmod.so': dlopen(/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod/libs/Rcppsbmod.so, 6): Library not loaded: libsundials_cvode.3.dylib Referenced from: /Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod/libs/Rcppsbmod.so Reason: image not found Error: loading failed` – Satya Mar 08 '18 at 23:56
  • I have checked `ls /usr/local/lib | grep sundials` gives `libsundials_cvode.1.0.0.dylib libsundials_cvode.1.dylib libsundials_cvode.3.1.0.dylib libsundials_cvode.3.dylib libsundials_cvode.a libsundials_cvode.dylib libsundials_cvodes.2.0.0.dylib libsundials_cvodes.2.dylib libsundials_cvodes.3.1.0.dylib libsundials_cvodes.3.dylib libsundials_cvodes.a libsundials_cvodes.dylib` – Satya Mar 08 '18 at 23:57
  • @SN248 How did you install the SUNDIALS library? – Ralf Stubner Mar 09 '18 at 05:48
  • Installing SUNDIALS on OS X and Linux I think is not so hard. I am specifically thinking about Windows. In any case, I do want users to be able to use this package even if they don't have SUNDIALS installed. I have seen this being done (nloptr, RCPPCAF), and will try to replicate whats being done there. Thanks. – Satya Mar 09 '18 at 20:00
  • Ralf Stubner - Thanks for your answer. I haven't tried your suggestion, but I was able to build the static library from source files and compile the package on my OS X machine. Now I am trying to do the same for Windows platform.Thanks again! – Satya Mar 11 '18 at 21:14