Why does computational time increase when using more than four cores on this bootstrapping model in R?

Question

When running the code snippet below for a bootstrapping exercise in R (version 2.15) on a x86_64-redhat-linux-gnu machine with 64 cores the observed computational times seem to degrade after using more than 4 cores. Are there additional configuration changes that are needed for R to make use of more than 4 cores?

library(doMC)
library(plyr)
df <- empty <- data.frame(cores = numeric(), trial = numeric(), time = numeric())

for(i in 1:64){
     registerDoMC(cores=i)
     trials <- 1000
     stime <- system.time({
              r <- foreach(icount(trials), .combine=cbind) %dopar% {
                   ind <- sample(1000000, 1000000, replace=TRUE)
                   mean(ind)}})
     df<- rbind(df, data.frame(cores=i,trial=1,time=stime[3]))
}
df2 <- ddply(df, c("cores"), summarize, time_avg = mean(time))

Below is a breakdown of processing time by number of cores.

num_cores time(seconds)
1   16.544
2   8.198
3   5.627
4   4.313
5   7.045
6   8.898
7   10.412
8   11.539
9   12.382
10  13.329
11  13.786
12  14.375
13  14.977
14  15.095
15  14.984
16  15.393
17  15.728
18  15.983
19  16.039
20  15.947
21  16.101
22  16.365
23  16.549
24  16.687
25  17.022
26  17.116
27  17.212
28  17.548
29  17.605
30  17.672
31  18.067
32  18.158
33  16.884
34  17.2
35  17.167
36  17.178
37  17.516
38  17.425
39  17.449
40  17.845
41  17.758
42  17.74
43  18.093
44  14.481
45  14.25
46  18.441
47  18.294
48  18.311
49  18.694
50  18.692
51  15.936
52  16.495
53  16.512
54  18.627
55  19.019
56  18.631
57  13.916
58  19.227
59  19.225
60  13.606
61  18.029
62  19.392
63  19.378
64  19.753

This post may be relevant: http://stackoverflow.com/questions/14614306/why-is-the-parallel-package-slower-than-just-using-apply/14614734#14614734 — Arun, Apr 02 '13 at 21:34
How about http://stackoverflow.com/questions/3547831/different-behavior-when-using-different-number-of-multicoring-workers ? You can't use more cores than you have. — Roman Luštrik, Apr 02 '13 at 21:38

score 2 · Answer 1 · answered Apr 03 '13 at 15:22

I don't think it's a matter of configuring R: the tasks are just so small that you can't take advantage of many cores. From the "1 core" time, each task is taking less than 16.5 milliseconds, and probably a lot less. The foreach package wasn't intended for that kind of fine-grained problem.

To have any hope, you could try using explicit chunking:

r <- foreach(n=idiv(trials, chunks=getDoParWorkers()), .combine='c') %dopar% {
  sapply(seq_len(n), function(i) mean(sample(1000000, 1000000, replace=TRUE)))
}

I don't know what your goal is, but I would use mclapply directly for this kind of problem.

Why does computational time increase when using more than four cores on this bootstrapping model in R?

1 Answers1