0

I recently changed my R BLAS framework to vecLib, which ships with Mac. After doing so, I had problems with the parallel package in R. Here is an example:

library(parallel)
xx1<-matrix(runif(2*70),ncol=2)
mcl.test<-mclapply(1:2,function(i) sum(tcrossprod(xx1[,i])),mc.cores = 2)
mcl.test

[[1]]
NULL

[[2]]
NULL

This (mclapply returns NULL randomly) discussion says that mclapply may be returning NULL because of memory issues. While a 70x70 matrix shouldn't cause memory issues (my workspace is otherwise empty, garbage collected), note that the problem goes away for a 60x60 matrix:

xx2<-matrix(runif(2*60),ncol=2)
mcl.test<-mclapply(1:2,function(i) sum(tcrossprod(xx2[,i])),mc.cores = 2)
mcl.test

[[1]]
[1] 754.1371

[[2]]
[1] 889.7769

I have no problem when mc.cores=1. I also have no problem if I switch my BLAS back to the default. Further, there is no problem when I use R from the terminal. What is the right way to make multithreaded R (via vecLib) work with mclapply outside the terminal?

sessionInfo()

R version 3.4.3 (2017-11-30)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_3.4.3 tools_3.4.3
Devin F
  • 1
  • 2

1 Answers1

0

The only way I found to make this work was to force vecLib to use only one thread (see this), by setting the environment variable export VECLIB_MAXIMUM_THREADS=1. This was not totally satisfying, because the sum of threads used by BLAS and the cores used by mclapply could still be less than the total number of available cores.

Hence, I switched to openBLAS instead of vecLib. It appears that openBLAS does not have this problem (though it may have in the past). With openBLAS, it appears I can run something on many cores using mclapply, with each child process using multithreading as much as possible.

Devin F
  • 1
  • 2