Stack Overflow
Stack Overflow

Questions tagged [domc]

The doMC package is a parallel backend for the foreach package. It provides a mechanism needed to execute foreach loops in parallel.

The doMC package is a parallel backend for the foreach package. It provides a mechanism needed to execute foreach loops in parallel. The foreach package must be used in conjunction with a package such as doMC in order to execute code in parallel. The user must register a parallel backend to use, otherwise foreach will execute tasks sequentially. The doMC package acts as an interface between foreach and the multicore package.

Resources:

75 questions
51
votes
1 answer

the difference between doMC and doParallel in R

What's the difference between doParallel and doMC in R concerning foreach function? doParallel supports windows, unix-like, while doMC supports unix-like only. In other words, why doParallel cannot replace doMC directly? Thank…
Zhilong Jia
  • 2,329
  • 1
  • 22
  • 34
29
votes
1 answer

doMC vs doSNOW vs doSMP vs doMPI: why aren't the various parallel backends for 'foreach' functionally equivalent?

I've got a few test pieces of code that I've been running on various machines, always with the same results. I thought the philosophy behind the various do... packages was that they could be used interchangeably as a backend for foreach's %dopar%. …
Zach
  • 29,791
  • 35
  • 142
  • 201
22
votes
5 answers

R multicore mcfork(): Unable to fork: Cannot allocate memory

I'm getting the titular error: mcfork(): Unable to fork: Cannot allocate memory after trying to run a function with mcapply, but top says I'm at 51% This is on an EC2 instance, but I do have up-to-date R. Does anyone know what else can cause this…
N. McA.
  • 4,796
  • 4
  • 35
  • 60
12
votes
3 answers

Is it possible to get a progress bar with foreach and a "multicore-kind" of backend

While using "multicore" parallelism using foreach and the doMC backend (I use doMC as at the time I looked into it other package did not allow logging from the I would like to get a progress bar, using the progress package, but any progress (that…
statquant
  • 13,672
  • 21
  • 91
  • 162
12
votes
3 answers

package doMC NOT available for R version 3.0.0 warning in install.packages

For some packages like doMC & doSMP, i get the warning & inability to library(doMC). As shown below, i have no problem with subselect thus no file/directory permission issue. Also tried repo=http://cran.us.r-project.org & others, no luck. please…
Yu Le
  • 233
  • 1
  • 4
  • 8
10
votes
3 answers

how to use %dopar% when only import foreach in DESCRIPTION of a package

How to avoid "could not find function "%dopar%"" in a function of a package when only imports (not depends) foreach in DESCRIPTION of a package? is there a way like foreach::%dopar% as I use foreach::foreach in function? Thank you. Code like: In…
Zhilong Jia
  • 2,329
  • 1
  • 22
  • 34
9
votes
1 answer

How to use doMC under Windows or alternative parallel processing implementation for glmnet?

I am on Win7 OS with R 3.3.1 in Rstudio. Intention is to use glmnet with parallel processing. From the ?glmnet help: parallel: If TRUE, use parallel foreach to fit each fold. Must register parallel before hand, such as doMC or others. See the …
tomka
  • 2,516
  • 7
  • 31
  • 45
7
votes
4 answers

How to kill a doMC worker when it's done?

The documentation for doMC seems very sparse, listing only doMC-package and registerDoMC(). The problem I'm encountering is I'll spawn several workers via doMC/foreach, but then when the job is done they just sit there taking up memory. I can go and…
Patrick McCarthy
  • 2,478
  • 2
  • 24
  • 40
6
votes
1 answer

R foreach not using multiple cores

I have a case where foreach using doMC as a backend produces different behaviors on different machines. On a linux server running Ubuntu 12.04.4 LTS the following code (adapted from the foreach vingette) runs 5 jobs simultaneously on a single…
sahoang
  • 375
  • 3
  • 9
5
votes
1 answer

Allow foreach workers to register and distribute sub-tasks to other workers

I have an R code that involves several foreach workers to perform some tasks in parallel. I am using foreach and doMC for this purpose. I want to let each of the foreach workers recruits some new workers and distribute some parts of their code,…
imriss
  • 1,815
  • 4
  • 31
  • 46
5
votes
1 answer

Ubuntu R ForEach / DoMC not using multiple cores

I have built a function in R (running on Ubuntu 12.04 LTS 64bit, 4 core i7 server with multithreading and 6gb ram) where I've installed R using the standard packages: sudo apt-get install r-base r-recommended r-base-dev sudo apt-get install…
Carst
  • 1,614
  • 3
  • 17
  • 28
4
votes
0 answers

dopar glmnet failing silently

I'm using glmnet to fit some models and am cross-validating for lambda. I'm using cv.glmnet by default (since it does complete cross-validation of lambda internally), but below I focus on the first step of that function, which is the one causing…
MichaelChirico
  • 33,841
  • 14
  • 113
  • 198
4
votes
1 answer

Parallel query of SQLite database in R

I have a large database (~100Gb) from which I need to pull every entry, perform some comparisons on it, and then store the results of those comparisons. I have attempted to run parallel queries within a single R sessions without any success. I can…
alexvpickering
  • 632
  • 1
  • 8
  • 20
4
votes
3 answers

%dopar% error after several iterations

I've been trying to get a parallelized foreach loop running in R, it works fine for approximately ten iterations but then crashes, showing the error: Error in { : task 7 failed - "missing value where TRUE/FALSE needed" Calls: %dopar% ->…
JohnnyTooBad
  • 77
  • 2
  • 6
4
votes
0 answers

caret package is not using all the registered cores, using 'nnet' method for training

I am using the train() function of caret package with method='nnet', and I have registered 6 cores using doMC. But it uses only one core. This is my code: library(caret) library(foreach) library(doMC) registerDoMC(cores = 6) .... some…
Abhishek
  • 3,337
  • 4
  • 32
  • 51
1
2 3 4 5