Installing sf package in R under renv on Macos

Question

I'm having problems in installing the sf package through renv. I have installed R from source in my Mac and everything is fine. I was able to install rgdal and sf through install.packages but can't make it with renv. The problem is that the installation script can't find the header proj_api.h even if I indicated the full path.

With

install.packages("sf")

everything works without further intervention. Within renv:

> renv::activate()
> renv::install("sf")
....
checking proj_api.h usability... no
checking proj_api.h presence... no
checking for proj_api.h... no
configure: error: proj_api.h not found in standard or given locations.
ERROR: configuration failed for package ‘sf’
....

The proj_api.h header is under /usr/local/include/, so following the documentation I tried:

options(configure.args = c(sf="--with-proj-include=/usr/local/include/"))

before installing, but with the same result. I have no idea of what to try next. I don't understand why with renv paths are not found and why without they are (locations are pretty standard). Can't understand also how is it possible that the header is not found even if I pass the full path.

The environment variables without renv:

__CF_USER_TEXT_ENCODING
                        0xAF76729:0x0:0x0
_CE_CONDA               
_CE_M                   
CLICOLOR                1
COLUMNS                 117
CONDA_EXE               /Users/52488/miniconda3/bin/conda
CONDA_PYTHON_EXE        /Users/52488/miniconda3/bin/python
CONDA_SHLVL             0
DISPLAY                 /private/tmp/com.apple.launchd.heGiCHqO77/org.xquartz:0
DYLD_FALLBACK_LIBRARY_PATH
                        /Users/52488/Rinstallation/R-4.0.4/lib:/Library/Java/JavaVirtualMachines/jdk-15.0.2.jdk/Contents/Home/lib/server
EDITOR                  vi
HOME                    /Users/52488
LC_ALL                  en_US.UTF-8
LC_CTYPE                UTF-8
LINES                   30
LN_S                    ln -s
LOGNAME                 52488
LSCOLORS                gxBxhxDxfxhxhxhxhxcxcx
MAKE                    make
PAGER                   /usr/bin/less
PATH                    /Users/52488/miniconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware
                        Fusion.app/Contents/Public:/opt/X11/bin
PWD                     /Users/52488
R_ARCH                  
R_BROWSER               /usr/bin/open
R_BZIPCMD               /usr/bin/bzip2
R_DOC_DIR               /Users/52488/Rinstallation/R-4.0.4/doc
R_GZIPCMD               /usr/bin/gzip
R_HOME                  /Users/52488/Rinstallation/R-4.0.4
R_INCLUDE_DIR           /Users/52488/Rinstallation/R-4.0.4/include
R_LIBS_SITE             
R_LIBS_USER             ~/Library/R/4.0/library
R_PAPERSIZE             a4
R_PDFVIEWER             /usr/local/bin/evince
R_PLATFORM              x86_64-apple-darwin17.0
R_PRINTCMD              lpr
R_RD4PDF                times,inconsolata,hyper
R_SESSION_TMPDIR        /var/folders/jb/vpf_9ht52zj_0qk_3r1f0dlw5gfss9/T//Rtmp40nNZi
R_SHARE_DIR             /Users/52488/Rinstallation/R-4.0.4/share
R_STRIP_SHARED_LIB      strip -x
R_STRIP_STATIC_LIB      strip -S
R_SYSTEM_ABI            macos,gcc,gxx,gfortran,gfortran
R_TEXI2DVICMD           /usr/local/bin/texi2dvi
R_UNZIPCMD              /usr/bin/unzip
R_ZIPCMD                /usr/bin/zip
SED                     /usr/bin/sed
SHELL                   /usr/local/bin/bash
SHLVL                   1
SSH_AUTH_SOCK           /private/tmp/com.apple.launchd.1OK1KoT1o5/Listeners
TAR                     /usr/bin/tar
TERM                    xterm-256color
TERM_PROGRAM            Apple_Terminal
TERM_PROGRAM_VERSION    433
TERM_SESSION_ID         2BA70166-937D-47AB-B6B6-23316D978294
TMPDIR                  /var/folders/jb/vpf_9ht52zj_0qk_3r1f0dlw5gfss9/T/
USER                    52488
XPC_FLAGS               0x0
XPC_SERVICE_NAME        0

With renv:

__CF_USER_TEXT_ENCODING
                        0xAF76729:0x0:0x0
_CE_CONDA               
_CE_M                   
CLICOLOR                1
COLUMNS                 117
CONDA_EXE               /Users/52488/miniconda3/bin/conda
CONDA_PYTHON_EXE        /Users/52488/miniconda3/bin/python
CONDA_SHLVL             0
DISPLAY                 /private/tmp/com.apple.launchd.heGiCHqO77/org.xquartz:0
DYLD_FALLBACK_LIBRARY_PATH
                        /Library/Frameworks/GDAL.framework/unix/lib:/Users/52488/Rinstallation/R-4.0.4/lib:/Library/Java/JavaVirtualMachines/jdk-15.0.2.jdk/Contents/Home/lib/server
EDITOR                  vi
HOME                    /Users/52488
LC_ALL                  en_US.UTF-8
LC_CTYPE                UTF-8
LINES                   30
LN_S                    ln -s
LOGNAME                 52488
LSCOLORS                gxBxhxDxfxhxhxhxhxcxcx
MAKE                    make
PAGER                   /usr/bin/less
PATH                    /Users/52488/miniconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware
                        Fusion.app/Contents/Public:/opt/X11/bin
PWD                     /Users/52488/Documents/staffing
R_ARCH                  
R_BROWSER               /usr/bin/open
R_BZIPCMD               /usr/bin/bzip2
R_DOC_DIR               /Users/52488/Rinstallation/R-4.0.4/doc
R_GZIPCMD               /usr/bin/gzip
R_HOME                  /Users/52488/Rinstallation/R-4.0.4
R_INCLUDE_DIR           /Users/52488/Rinstallation/R-4.0.4/include
R_LIBS_SITE             
R_LIBS_USER             /Users/52488/Documents/staffing/renv/library/R-4.0/x86_64-apple-darwin17.0:/private/var/folders/jb/vpf_9ht52zj_0qk_3r1f0dlw5gfss9/T/Rtmpm4f15L/renv-system-library
R_PAPERSIZE             a4
R_PDFVIEWER             /usr/local/bin/evince
R_PLATFORM              x86_64-apple-darwin17.0
R_PRINTCMD              lpr
R_RD4PDF                times,inconsolata,hyper
R_SESSION_TMPDIR        /var/folders/jb/vpf_9ht52zj_0qk_3r1f0dlw5gfss9/T//Rtmpm4f15L
R_SHARE_DIR             /Users/52488/Rinstallation/R-4.0.4/share
R_STRIP_SHARED_LIB      strip -x
R_STRIP_STATIC_LIB      strip -S
R_SYSTEM_ABI            macos,gcc,gxx,gfortran,gfortran
R_TEXI2DVICMD           /usr/local/bin/texi2dvi
R_UNZIPCMD              /usr/bin/unzip
R_ZIPCMD                /usr/bin/zip
RENV_DEFAULT_R_ENVIRON
                        <NA>
RENV_DEFAULT_R_ENVIRON_USER
                        <NA>
RENV_DEFAULT_R_LIBS     <NA>
RENV_DEFAULT_R_LIBS_SITE
                        
RENV_DEFAULT_R_LIBS_USER
                        ~/Library/R/4.0/library
RENV_DEFAULT_R_PROFILE
                        <NA>
RENV_DEFAULT_R_PROFILE_USER
                        <NA>
RENV_PROJECT            /Users/52488/Documents/staffing
RETICULATE_MINICONDA_PYTHON_ENVPATH
                        /Users/52488/Documents/staffing/renv/python/r-reticulate
SDKROOT                 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
SED                     /usr/bin/sed
SHELL                   /usr/local/bin/bash
SHLVL                   1
SSH_AUTH_SOCK           /private/tmp/com.apple.launchd.1OK1KoT1o5/Listeners
TAR                     /usr/bin/tar
TERM                    xterm-256color
TERM_PROGRAM            Apple_Terminal
TERM_PROGRAM_VERSION    433
TERM_SESSION_ID         34A286FD-8106-4F78-88F9-8913E45EDCDE
TMPDIR                  /var/folders/jb/vpf_9ht52zj_0qk_3r1f0dlw5gfss9/T/
USER                    52488
XPC_FLAGS               0x0
XPC_SERVICE_NAME        0

Three variables differ, but it's pretty normal I guess:

[1] "PWD"              "R_LIBS_USER"      "R_SESSION_TMPDIR"

Of course there are the RENV* variables which are only present in the renv case.

Digging further and taking inspiration from this question, I checked the include path of gcc. There are differences depending on renv.

With renv:

> system("echo | gcc -Wp,-v -x c++ - -fsyntax-only", intern=TRUE)
clang -cc1 version 12.0.0 (clang-1200.0.32.29) default target x86_64-apple-darwin19.6.0
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1
 /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
 /Library/Developer/CommandLineTools/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.

Without renv:

> system("echo | gcc -Wp,-v -x c++ - -fsyntax-only", intern=TRUE)
clang -cc1 version 12.0.0 (clang-1200.0.32.29) default target x86_64-apple-darwin19.6.0
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1
 /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include
 /Library/Developer/CommandLineTools/usr/include
 /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks (framework directory)
End of search list.

It is apparent that inside renv the standard /usr/local/include is not present (there are also some differences in the other paths). Why is that? How can I fix it?

Nice. I'm not in front of my computer, but if you compare them, (do a setdiff both ways), is there anything striking that pops out? — Sirius, Apr 01 '21 at 19:56
Made an edit. There are just 3 variables that differ, and it's pretty normal that they do. — nicola, Apr 01 '21 at 20:15
Yeah, I wouldn't suspect them. You could change them in the renv R session to match the other but I doubt it's related. What if you debug(install.packages) , and just before it runs the actual install, inspect the environment again, and see if anything else is different — Sirius, Apr 01 '21 at 20:37
@Sirius I made an edit with the search path of `gcc`. Do you have any idea on that? — nicola, Apr 02 '21 at 04:41
What if you untar the package tarball, do you find makefiles or build scripts? Maybe they're doing something unorthodox in terms of including headers and libs — Sirius, Apr 02 '21 at 08:42
Do other packages that require c compiling install under renv on your system? — Sirius, Apr 02 '21 at 08:44
I tried tarballing `sf` and running the configure script. Everything was ok. Within `renv` it looks like that some locations are not seen; however, for the `units` package for instance, by adding in the configure script `/usr/local/include/` everything worked (I didn't need to include the path explicitly in a normal R session). — nicola, Apr 02 '21 at 09:14
So, it looks to me that I have 2 issues. The first is that inside `renv` the C preprocessor doesn't look for the same paths as outside `renv`. The second is that the package `sf` for some reason ignores the `--with-proj-include` path provided. — nicola, Apr 02 '21 at 09:16
@Sirius Actually, I was a little sloppy in checking the environment variables. After loading `renv` a `SDKROOT` variable appears and that is the culprit. If I export it on a shell and then run `cpp -v`, the `/usr/local/include` path disappear. Now I'm trying to understand why that's the case and how I can work around it. If you have any hint, that would be great. — nicola, Apr 02 '21 at 12:43
What happens if you unset it in .Rprofile? or just your R session? I don't see it on my linux system — Sirius, Apr 02 '21 at 13:18
It's a variable that has effect only for macos. I don't know if it's critical for other aspects. The .Rprofile is set by renv and I'd like a solution that lets me use renv without much tweaking. — nicola, Apr 02 '21 at 13:43
adding a single Sys.setenv call to an existing .Rprofile is ok I'd say. I do it all the time. At least until this gets fixed. — Sirius, Apr 02 '21 at 16:23
@Sirius The point is that renv builds an .Rprofile for each project, so you have to remember to edit the .Rprofile each time you open a project. I end up setting it in bash_profile with a blank value, so that renv does not overwrite it. Thank you for your hints and comments, it was very helpful. — nicola, Apr 03 '21 at 06:31
Np! Yeah I've been using renv extensively. Figured not all projects needed sf, so editing only the few who needed it might work. But of course, env variables can be set multiple ways. Did renv developers respond? — Sirius, Apr 03 '21 at 08:08
Yes, you can follow here: https://github.com/rstudio/renv/issues/704 — nicola, Apr 03 '21 at 08:27

score 0 · Answer 1 · answered Nov 02 '21 at 15:15

I've went through this just now. I too had everything installed via install.packages('sf') and it worked just fine, until today I needed to work in renv environment.

This has helped me (assuming you have brew installed):

brew install gdal --HEAD
brew install gdal
brew install proj

I don't know why sf actually worked before, probably R installs these binaries someplace r-internal.

Installing sf package in R under renv on Macos

1 Answers1