Using "snow" parallel operations in bootstrap_parameters/model on merMod object (R)

Question

I've been using bootstrap_parameters (parameters package in R) on generalised linear mixed models produced using glmmTMB. These work fine without parallel processing (parallel = "no") and also works fine on my old and slow mac using parallel = "multicore". I'm working on a new PC (Windows OS) so need to use parallel = "snow" however I get the following error:

system.time(b <- bootstrap_parameters(m1, iterations = 10, parallel = "snow", n_cpus = 6)) Error in data.frame(..., check.names = FALSE) : arguments imply differing number of rows: 0, 1 In addition: Warning message: In lme4::bootMer(model, boot_function, nsim = iterations, verbose = FALSE, : some bootstrap runs failed (10/10) Timing stopped at: 0.89 0.3 7.11

If I select n_cpus = 1, the function works or if I feed bootstrap_parameters or bootstrap_model an lm object (where the underlying code uses boot::boot) it also works fine. I have narrowed the problem down to bootMer (lme4). I suspect the dataset exported using clusterExport is landing in an environment that is different from where clustered bootMer function is looking. The following is a reproduceable example

library(glmmTMB)
library(parameters)
library(parallel)
library(lme4)

m1 <- glmmTMB(count ~ mined + (1|site), zi=~mined,
              family=poisson, data=Salamanders)
summary(m1)

cl <- makeCluster(6)
clusterEvalQ(cl, library("lme4"))
clusterExport(cl, varlist = c("Salamanders"))

system.time(b <- bootstrap_parameters(m1, iterations = 10, parallel = "snow", n_cpus = 6))

stopCluster(cl)

Any ideas on solving this problem?

Answer 1

You need to clusterEvalQ(cl, library("glmmTMB")). From https://github.com/glmmTMB/glmmTMB/issues/843:

This issue is more or less resolved by a documentation patch (we need to explicitly clusterEvalQ(cl, library("glmmTMB"))). The only question is whether we can make this any easier for users. There are two problems here: (1) when the user sets up their own cluster rather than leaving it to be done in bootMer, more explicit clusterEvalQ/clusterExport stuff is necessary in any case; (2) bootMer internally does parallel::clusterExport(cl, varlist=getNamespaceExports("lme4")) if it is setting up the cluster (not if the cluster is set up and passed to bootMer by the user), but we wouldn't expect it to extend the same courtesy to glmmTMB ...

For example, this all works:

library(glmmTMB)
library(parameters)
library(parallel)
library(lme4)

m1 <- glmmTMB(count ~ mined + (1|site), zi=~mined,
              family=poisson, data=Salamanders)
summary(m1)

par_cores <- max(1, floor(detectCores()/2))
par_cluster <- makeCluster(rep("localhost", par_cores), outfile = "log.txt")

clusterEvalQ(par_cluster, library("lme4"))
clusterExport(par_cluster, varlist = c("Salamanders"))

system.time(b <- bootstrap_parameters(m1, iterations = 10, cluster = par_cluster))

system.time(b2 <- bootMer(m1,
                          FUN = function(x)fixef(x)$cond,
                          nsim = 10, cl = par_cluster))
stopCluster(par_cluster)