19

Installing R packages that need compiling in macOS is broken since Big Sur. Here is such an example.

> install.packages('nlme')

  There is a binary version available but the source version is later:
      binary  source needs_compilation
nlme 3.1-150 3.1-151              TRUE

Do you want to install from sources the package which needs compilation? (Yes/no/cancel) Yes
installing the source package ‘nlme’

trying URL 'https://cran.rstudio.com/src/contrib/nlme_3.1-151.tar.gz'
Content type 'application/x-gzip' length 805592 bytes (786 KB)
==================================================
downloaded 786 KB

* installing *source* package ‘nlme’ ...
** package ‘nlme’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
"gfortran-4.8" -fno-optimize-sibling-calls  -fPIC  -Wall -g -O2  -c chol.f -o chol.o
gfortran-4.8: warning: couldn’t understand kern.osversion ‘20.1.0
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c corStruct.c -o corStruct.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c gnls.c -o gnls.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c init.c -o init.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c matrix.c -o matrix.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c nlOptimizer.c -o nlOptimizer.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c nlme.c -o nlme.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c nlmefit.c -o nlmefit.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c pdMat.c -o pdMat.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c pythag.c -o pythag.o
"gfortran-4.8" -fno-optimize-sibling-calls  -fPIC  -Wall -g -O2  -c rs.f -o rs.o
gfortran-4.8: warning: couldn’t understand kern.osversion ‘20.1.0
clang -mmacosx-version-min=10.13 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o nlme.so chol.o corStruct.o gnls.o init.o matrix.o nlOptimizer.o nlme.o nlmefit.o pdMat.o pythag.o rs.o  -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: framework not found CoreFoundation
clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [nlme.so] Error 1
ERROR: compilation failed for package ‘nlme’
* removing ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/nlme’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/nlme’
Warning in install.packages :
  installation of package ‘nlme’ had non-zero exit status

The downloaded source packages are in
    ‘/private/var/folders/_3/b8whcf8d1bb8w_lr2rrscb5m0000gp/T/RtmpWCjwfs/downloaded_packages’

I've tried reinstalling xcode and Command Line Tools and completed Brew update + upgrade. Any suggestions appreciated.

geotheory
  • 22,624
  • 29
  • 119
  • 196
  • Just spotted this (ignored) question which looks related https://stackoverflow.com/questions/65047685/r-package-installation-fail-ld-framework-not-found-corefoundation-clang-7-err – geotheory Dec 11 '20 at 13:17
  • 2
    Lots of other people stumbling upon the same issue. https://pbs.twimg.com/media/EbLLgorUcAEpWYf?format=png&name=900x900 – IRTFM Dec 13 '20 at 21:20
  • Thanks @IRTFM. Though very fristrating! – geotheory Dec 13 '20 at 23:07
  • Might be better to communicate with the R-MAC-sig mailing list. – IRTFM Dec 14 '20 at 07:24
  • 1
    I'm stuck with the same problem. I've tried `r-macos-rtools` but the installation does not work for Big Sur. Here is some info on how to fix the installation, though I was not able to: https://github.com/rmacoslib/r-macos-rtools/issues/42 – epsilone Dec 15 '20 at 19:58

2 Answers2

33

This has been challenging, but here are the steps I used compile R packages from source on MacOS Big Sur:

  1. Reinstall xcode command line tools

(don't believe Software Update if it says 'up to date' - he lies - brew doctor said my version was actually old)

sudo rm -rf /Library/Developer/CommandLineTools
sudo xcode-select --install
  1. Install gcc & llvm via Homebrew (instructions for installing Homebrew) or, if you already have gcc/llvm installed, skip to the next step
# WARNING: This can take several hours
brew install gcc
brew install llvm
  1. If you already have gcc & llvm installed via Homebrew:
brew cleanup
brew update
brew upgrade
brew reinstall gcc
brew reinstall llvm
  1. Link some headers into /usr/local/include
sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/* /usr/local/include/

# I believe you can safely ignore warnings like this:
#ln: /usr/local/include//tcl.h: File exists
#ln: /usr/local/include//tclDecls.h: File exists
#ln: /usr/local/include//tclPlatDecls.h: File exists
#ln: /usr/local/include//tclTomMath.h: File exists
#ln: /usr/local/include//tclTomMathDecls.h: File exists
#ln: /usr/local/include//tk.h: File exists
#ln: /usr/local/include//tkDecls.h: File exists
#ln: /usr/local/include//tkPlatDecls.h: File exists
  1. Edit your ~/.R/Makevars file to include only these lines:
LOC=/usr/local/gfortran
CC=$(LOC)/bin/gcc -fopenmp
CXX=$(LOC)/bin/g++ -fopenmp
CXX11 = $(LOC)/bin/g++ -fopenmp

CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L$(LOC)/lib -Wl,-rpath,$(LOC)/lib,-L/usr/local/lib
CPPFLAGS=-I$(LOC)/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/usr/local/include

FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin19/10.2.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm
CXX1X=/usr/local/gfortran/bin/g++
CXX98=/usr/local/gfortran/bin/g++
CXX11=/usr/local/gfortran/bin/g++
CXX14=/usr/local/gfortran/bin/g++
CXX17=/usr/local/gfortran/bin/g++
  1. Compile a package from source in R/Rstudio
# Compile the nlme package from source
install.packages("nlme", type = "source")

# To check whether openmp is enabled, compile data.table:
install.packages("data.table", type = "source")
jared_mamrot
  • 22,354
  • 4
  • 21
  • 46
  • 1
    Thanks Jared, I get `$ git -C "/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core" fetch --unshallow fatal: --unshallow on a complete repository does not make sense` Any idea why? – geotheory Dec 17 '20 at 13:38
  • My homebrew is currently a mess due to QGIS, but hopefully once I've resolved that in next day or so your solution will work. – geotheory Dec 18 '20 at 00:38
  • 1
    Brew fixed and your workflow fixed this install. Many thanks :) – geotheory Dec 20 '20 at 00:44
  • After doing all this I still get the error about `ld: framework not found CoreFoundation` – Florag Dec 24 '20 at 09:32
  • Trying to figure out why it didn't work.... one thing is that my version of gfortran seems to be using the libraries of a different gcc version to that specified in the Makevars file above. Mine has `/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0`. Changing the makevars to use that doesn't help. – Florag Dec 24 '20 at 09:55
  • Thanks for responding! Have now gone through it all again both in whole, and rerunning `brew reinstall gcc` again after seeing your message.. I still have the x86_64-apple-darwin15/6.1.0 folder there. Any suggestions would be very, very welcome! – Florag Dec 24 '20 at 12:31
  • Just noticed that when I run `reinstall gcc` it seems to pour it to `/usr/local/Cellar/gcc/10.2.0`. Any idea why it would use this address instead? – Florag Dec 24 '20 at 13:53
  • 4
    After all steps and restarting I got `ld: framework not found CoreFoundation`. Installing fresh gfortran for R/Mac worked for me: https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/10.2-bigsur-intel – speleo Dec 29 '20 at 21:45
  • 1
    @speleo, that's it! Installing gfortran from that link solved it for me. Was a bit hesitant given that the repo is just from a researcher, but it's got my R working again, so many, many thanks! – Florag Jan 03 '21 at 12:01
  • 1
    There's actually an alternative suggestion to installing that gfortran link. That is, to use the version of gfortran that brew provides with gcc, but I think this requires different makevars lines to those provided above. See this link: https://github.com/Homebrew/homebrew-cask/issues/96135 – Florag Jan 03 '21 at 13:41
  • Glad you got it working @Florag! Sorry I was offline over Christmas/New Year's and couldn't help you solve the issue. My understanding was that `brew reinstall gcc` linked brew's 'gcc' fortran to /usr/local/gfortran but apparently that's not the case on other people's systems - thanks @speleo for the additional troubleshooting! – jared_mamrot Jan 03 '21 at 22:37
  • @speleo and @jared_mamrot, you both saved me, thank you! I get the message `ld: warning: dylib (/usr/local/opt/llvm/lib/libunwind.dylib) was built for newer macOS version (11.0) than being linked (10.16)` after using the fresh gfortran linked. Do you also? – user551504 Feb 02 '21 at 03:04
  • 1
    @user551504 that warning is related to the statically linked llvm libraries. You can avoid the warning by removing llvm (`brew uninstall llvm`) and the references to llvm from the .R/Makevars file, and many packages will still compile but some will fail. Unfortunately, there are still some packages that I couldn't get to compile after following these steps - e.g. rstan - but the majority of packages compile successfully. – jared_mamrot Feb 04 '21 at 23:19
  • @jared_mamrot what if it says `Error: unexpected symbol in "sudo rm"` in the first step? – nilsinelabore Apr 20 '21 at 01:04
0

MacOS Ventura 13.5
Found and tried many different solutions for multiple hours, also the one presented above. The solution for me was to remove the problematic directory from the Terminal.

cd /Library/Developer/CommandLineTools/SDKs
rm -rf MacOSX.sdk

If you are unsure, you can rename the directory, instead of deleting it

cd /Library/Developer/CommandLineTools/SDKs
sudo mv MacOSX.sdk MacOSX_old.sdk
obruzzi
  • 456
  • 1
  • 4
  • 12