Use doParallel with Rcpp function on window inside an R package

Question

I undertand from this post that the easiest way to use Rcpp functions with doParallel on window is to build a package. I am currently building an R package and I need to parallelise some tasks in the package. I have an Rcpp function and an R function that performs a loop on the Rcpp function. These functions will be in my package. How can I construct the R function? According to the post above, It will be easier to build a first package with the Rcpp function an call this package in the second package of the R function. Is there a way to put the both functions in the same package?

Answer 1

Yes, you can put as many functions as you want within the package. The reason for suggesting everything is in an R package is because you would otherwise have to compile the code on every thread or node that you spin your code up on. This is because the Rcpp functions are compiled locally and only have a thread-specific pointer reference. In particular, see the discussion in: Using Rcpp functions inside of R's par*apply functions from the parallel package.

Sample package would be:

https://github.com/r-pkg-examples/rcpp-and-doparallel

In particular, the R function should correctly setup and teardown the parallel backend.

mean_parallel_compute = function(n, mean = 0, sd = 1,
                                 n_sim = 1000,
                                 n_cores = parallel::detectCores()) {

  # Construct cluster
  cl = parallel::makeCluster(n_cores)

  # After the function is run, close the cluster.
  on.exit(parallel::stopCluster(cl))

  # Register parallel backend
  doParallel::registerDoParallel(cl)

  # Compute estimates
  estimates = foreach::foreach(i = iterators::icount(n_sim), # Perform n simulations
                               .combine = "rbind",           # Combine results
                                                             # Self-load
                               .packages = "Rcpp2doParallel") %dopar% {
    random_data = rnorm(n, mean, sd)

    result = mean_rcpp(random_data) # or use Rcpp2doParallel::mean_rcpp()
    result
  }

  estimates
}

To pass R CMD check make sure to have the following roxygen2 import tags:

#' @importFrom foreach %dopar% foreach
#' @importFrom iterators icount
#' @importFrom doParallel registerDoParallel

In addition, make sure DESCRIPTION has the following:

LinkingTo: 
    Rcpp
Imports: 
    doParallel,
    Rcpp,
    foreach,
    iterators,
    parallel

Some other examples:

• Compiling Rcpp functions using ClusterEvalQ
• Using Rcpp within parallel code via snow to make a cluster