R, error 'could not find function' in foreach with timeOut

Question

I'm tring to insert in a foreach a timeout control like this:

#Parallel function
runnable_X<-function(ID) {
  require(R.utils)
  Sys.sleep(ID)
  return(ID)
}

#foreach function with timeout
foreach_timeOut<-function() {
  tryCatch({
    require(R.utils)
    withTimeout({

      out_list<-foreach(ID=c(1:20),.options.multicore=list(preschedule=FALSE)) %dopar% runnable_X(ID)

    },
    timeout=5); ### Cumulative Timeout for entire process
  }, TimeoutException=function(ex) {
    return("Time Out!")
  })
} 


library(doParallel)
require(R.utils)

#Parallel registration
registerDoParallel()

#NUmber of cores
options(cores=5)

foreach_timeOut()

I have this error:

Error in runnable_X(ID) : 
  task 1 failed - "could not find function "runnable_X""

If I declare runnable_X function inside foreach_timeout this doesn't happen, but I can't do this.

score 1 · Accepted Answer · answered Feb 07 '18 at 09:17

1

If you use built in functions, you have to add them to the cluster:

N_CORES <- 5   
cl <- makeCluster(N_CORES, outfile = "")
registerDoParallel(cl)
clusterEvalQ( cl, {

  runnable_X <- function() { stuff }

})

answered Feb 07 '18 at 09:17

karen

822
1
6
22

I use the cluster created automatically by registerDoParallel, I don't use makeCluster – Terru_theTerror Feb 07 '18 at 09:22
I see, although you would solve your problema by adding the line to create the cluster, and it wouldn't do any difference to the code – karen Feb 07 '18 at 09:27
This is just a "toy code", for other reason I don't want create the cluster. If I have this function inside an ad hoc package, the function will be visible inside the foreach? – Terru_theTerror Feb 07 '18 at 09:42
If this function is insight a package you should type: foreach(x, .packages = c("package_adhoc")) %dopar% – karen Feb 07 '18 at 12:13

score 1 · Answer 2 · answered Feb 07 '18 at 10:53

Thanks to @karen for the help (+1) in case we have a cluster, but I have found this solution more fitting to my situation.

#Parallel function
runnable_X<-function(ID) {
  require(R.utils)
  Sys.sleep(ID)
  return(ID)
}

I will add in foreach export command with my function

#foreach function with timeout
foreach_timeOut<-function() {
  tryCatch({
    require(R.utils)
    withTimeout({

      out_list<-foreach(ID=c(1:20),
                        .options.multicore=list(preschedule=FALSE),
                        .export = c("runnable_X")
                        ) %dopar% runnable_X(ID)

    },
    timeout=5); ### Cumulative Timeout for entire process
  }, TimeoutException=function(ex) {
    return("Time Out!")
  })
} 


library(doParallel)
require(R.utils)

#Parallel registration
registerDoParallel()

#NUmber of cores
options(cores=5)

foreach_timeOut()
[1] "Time Out!"

R, error 'could not find function' in foreach with timeOut

2 Answers2