9

There's probably a gazillion threads on OSX+Rcpp+openMP, but the bottom line right now appears to be this (per coatless):

Unfortunately, with R 4.0.0 the CRAN distributed version of R loses the ability to use OpenMP without a custom setup.

I came across other ideas, including compiling llvm yourself, using homebrew or macports to install R and/or llvm and/or gcc, and then figuring out how to use the right compiler and/or flags with (R)cpp. However, I find this all very confusing.

I am not a mac user, but it seems to me that setting up a mac to compile Rcpp packages or code snippets with openMP seems to be too difficult for most mac users. However, I would like my R package on github to be used by more users, and since it relies on openMP, I am losing that audience.

Can someone provide the necessary steps to set up R on mac in a way that it can compile Rcpp code with openMP? I'd like to turn that into a quick tutorial.

EDIT: I should have added - on Apple Silicon, because there are some extra confusions where things go - /usr/local vs /opt

inferator
  • 474
  • 3
  • 12
  • 1
    For others who find their way here: I have written up instructions appropriate for both Big Sur _and_ Monterey [here](https://stackoverflow.com/questions/70638118). The details on Monterey aren't fundamentally different but there are some complications. – Mikael Jagan Jan 14 '22 at 03:35
  • Thank you Mikael! I haven't tried it yet, specifically because I expected new complications ;-) – inferator Jan 17 '22 at 20:56

2 Answers2

13

I spent a day figuring this out (original post here); here are the steps I used to compile R packages from source with openMP:

  1. Install xcode from the app store (instructions for installing xcode) then install/reinstall the xcode command line tools from the terminal:
# To delete an existing command line tools installation:
sudo rm -rf /Library/Developer/CommandLineTools

# To install the command line tools
sudo xcode-select --install
  1. Install gcc via Homebrew (instructions for installing Homebrew) or, if you already have gcc installed, skip to step 3.
# WARNING: This can take several hours
brew install gcc
  1. To avoid "legacy" version issues:
brew cleanup
brew update
brew upgrade
brew reinstall gcc
  1. Link some headers into /usr/local/include
sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/* /usr/local/include/

# You can ignore warnings like this:
#ln: /usr/local/include//tcl.h: File exists
#ln: /usr/local/include//tclDecls.h: File exists
#ln: /usr/local/include//tclPlatDecls.h: File exists
#ln: /usr/local/include//tclTomMath.h: File exists
#ln: /usr/local/include//tclTomMathDecls.h: File exists
#ln: /usr/local/include//tk.h: File exists
#ln: /usr/local/include//tkDecls.h: File exists
#ln: /usr/local/include//tkPlatDecls.h: File exists
  1. Check your version of gfortran (cd /usr/local/gfortran/lib/gcc/x86_64-apple-darwin19/; ls) then edit your ~/.R/Makevars file (if you don't have a file called Makevars in your ~/.R/ directory) and include only these lines:
LOC = /usr/local/gfortran
CC=$(LOC)/bin/gcc -fopenmp
CXX=$(LOC)/bin/g++ -fopenmp
CXX11 = $(LOC)/bin/g++ -fopenmp

CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L$(LOC)/lib -Wl,-rpath,$(LOC)/lib
CPPFLAGS=-I$(LOC)/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include

# (check that the version of gfortran - in this case 10.2.0 - matches the version specified in FLIBS)
FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin19/10.2.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm
CXX1X=/usr/local/gfortran/bin/g++
CXX98=/usr/local/gfortran/bin/g++
CXX11=/usr/local/gfortran/bin/g++
CXX14=/usr/local/gfortran/bin/g++
CXX17=/usr/local/gfortran/bin/g++
  1. Open R and install a package to test that it compiles with openMP enabled (when asked, compile from source = "Yes"):
install.packages("data.table", type = "source")

Unfortunately, I do not believe a more "simple" setup exists.

jared_mamrot
  • 22,354
  • 4
  • 21
  • 46
  • Thank you! Sorry for missing the original post. Also, why brew and not macports? – inferator Jul 06 '21 at 00:41
  • 2
    You are using Homebrew. Can you comment on whether you think this will succeed with the "standard" R provided by CRAN? (The usual advice from Simon Urbanek regarding Homebrew (or MacPorts for that matter) is "don't do that".) – IRTFM Jul 06 '21 at 00:42
  • My general advice regarding OSX is "don't use it". However, it's too late for that, so we have to make do with what people have. I am trying to follow jared's steps, but gfortran seems to be in a different directory. Homebrew apparently puts stuff under /opt instead of /usr/local where it belongs. – inferator Jul 06 '21 at 01:06
  • And I just came across another tutorial https://yiqingxu.org/public/BigSurError.pdf – inferator Jul 06 '21 at 01:07
  • When you install/reinstall gcc I believe it moves gfortran to /usr/local. On my original post some folks had the same issue and solved it by just installng gfortran from e.g. https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/10.2-bigsur-intel – jared_mamrot Jul 06 '21 at 01:10
  • Yes, that would install gfortran. However, I thought gcc is part of that gfortran installer. Wouldn't that overwrite/conflict with the installation from brew? – inferator Jul 06 '21 at 01:23
  • Sorry - I wasn't clear - gfortran is bundled with gcc. When you install via homebrew (`brew install gcc`) it should install it (or symlink, I can't remember) into /usr/local/, but some people ran into trouble with that step and had to do a 'clean install' of gfortran. I don't believe this would overwrite the homebrew gfortran, because the homebrew gfortran wasn't in the 'right spot'. – jared_mamrot Jul 06 '21 at 01:29
  • I see. And why do we need llvm? I looks like your Makevars uses gcc, and not clang. So why is llvm needed? – inferator Jul 06 '21 at 01:31
  • Honestly, I can't remember - I had to use the llvm clang to compile specific packages when gcc failed, but I don't remember the specifics. Perhaps llvm provides required headers / libraries? I'll look into it and get back to you – jared_mamrot Jul 06 '21 at 01:39
  • Speakin of Simon, he seems to prefer the clang route: https://mac.r-project.org/openmp/ – inferator Jul 06 '21 at 02:31
  • Based on my previous .R/Makevars files, I used llvm to compile rstan with openMP enabled back in 2020 when gcc failed, but I suspect you could skip the llvm install. For your use-case (installing a single specific package) perhaps the https://mac.r-project.org/openmp/ route would be best - you don't really need to worry about compiling other packages - that would definitely be simpler – jared_mamrot Jul 06 '21 at 02:56
  • I followed mac.r-project, but it doesn't work either. The apple silicon gfortran installer installs into /usr/local instead of /opt/R/arm64, and manually untaring gfortran into the right place doesn't resolve it either, because somehow the compiler looks in opt/ and usr/local. Bottom line: don't use OSX for serious work. – inferator Jul 06 '21 at 16:06
  • Ohh - sorry - my answer won't work on Apple M1 silicon - I didn't realise this was your intended goal. And I agree with you on the macOS thing. I use two work laptops: my work-supplied one (macOS) and my own (linux) and I find myself using my own linux laptop pretty regularly when something causes errors on macOS... – jared_mamrot Jul 06 '21 at 23:39
  • 1
    It was my mistake, I didn't clearly state it. However, you really helped me understand a few challenges, and I eventually found a solution. It's quite a shame that Apple makes it so hard, the performance of the M1 is really good once you get it to work. – inferator Jul 07 '21 at 00:20
  • Just in case this is useful for others. I use Big Sur and tried to install data.table following your instruction, but while I was able to do every single step the `install.package("data.table")` never installed a version that utilises OpenMP. Only after using `install.packages("data.table", type = "source", repos = "https://Rdatatable.gitlab.io/data.table")`, was I able to install a multi threaded version. – krenz Jan 21 '22 at 21:00
  • Thank you @krenz! Yes, the install.packages command should include `type = "source"` - I've edited my post to fix that issue - thanks for your comment! – jared_mamrot Jan 22 '22 at 21:10
7

Eventually, I found a process that works on a M1 mac with Big Sur.

  • Head over to https://mac.r-project.org/, it contains most things you will need
  • Download and install R via R-4.1-branch.pkg. The CRAN version might also work, but I used the installer from mac.r-project.org, which required opening the osx security settings to allow the installation.
  • Install RStudio, start it, and let it install the developer tools. Alternatively, run sudo xcode-select --install in Terminal.
  • Head to https://mac.r-project.org/openmp/. Download openmp-11.0.1-darwin20-Release.tar.gz and install it (see Terminal commands below).
curl -O https://mac.r-project.org/openmp/openmp-11.0.1-darwin20-Release.tar.gz
sudo tar fvx openmp-11.0.1-darwin20-Release.tar.gz -C /
  • Now we need to add compiler flags so that clan uses openMP. In Terminal, create the Makevars file.
cd ~
mkdir .R
nano .R/Makevars

in nano, paste these additional compiler flags into the Makevars file:

CPPFLAGS += -Xclang -fopenmp
LDFLAGS += -lomp

Hit Control+O, Control+X to save and close

  • Head over to the gfortran page: https://github.com/fxcoudert/gfortran-for-macOS/releases
  • Use the installer gfortran-ARM-11.0-BigSur.pkg to install gfortran.
  • For some reason it appears to install in /usr/local/gfortran, but R expects it in /opt. The mac-R team likes to separate arm64 and intel related files. We could go and fix paths, or simply also install gfortran under /opt. Download the tar file gfortran-ARM-11.0-BigSur.tar.xz. You can use curl, or just download it and point tar in the command line to it.
cd /opt/R/arm64/
sudo mkdir gfortran
sudo tar -xzyf gfortran-ARM-11.0-BigSur.tar.xz -C /opt/R/arm64/

(replace gfortran-ARM-11.0-BigSur.tar.xz with /users/YOURUSERNAME/downloads/gfortran-ARM-11.0-BigSur.tar.xz)

Now it should work.

Not an expert in OSX, but doing this so others can figure out how to use my R package. I'd like to streamline the process some more, but wiping the mac, reinstalling osx and testing it takes so much time.

mkln
  • 14,213
  • 4
  • 18
  • 22
inferator
  • 474
  • 3
  • 12