Stack Overflow
Stack Overflow

Questions tagged [doparallel]

R package that is a “parallel backend” for the foreach package. It provides a mechanism needed to execute foreach loops in parallel.

453 questions
87
votes
1 answer

Inconsistent behaviour with tm_map transformation functions when using multiple cores

Another potential title for this post could be "When parallel processing in R, does the ratio between the number of cores, loop chunk size, and object size matter?" I have a corpus I am running some transformations on using the tm package. Since the…
Doug Fir
  • 19,971
  • 47
  • 169
  • 299
51
votes
1 answer

the difference between doMC and doParallel in R

What's the difference between doParallel and doMC in R concerning foreach function? doParallel supports windows, unix-like, while doMC supports unix-like only. In other words, why doParallel cannot replace doMC directly? Thank…
Zhilong Jia
  • 2,329
  • 1
  • 22
  • 34
21
votes
2 answers

Why does foreach %dopar% get slower with each additional node?

I wrote a simple matrix multiplication to test out multithreading/parallelization capabilities of my network and I noticed that the computation was much slower than expected. The Test is simple : multiply 2 matrices (4096x4096) and return the…
JustGettinStarted
  • 784
  • 7
  • 21
19
votes
3 answers

doParallel, cluster vs cores

What is the difference between cluster and cores in registerDoParallel when using doParallel package? Is my understanding correct that on single machine these are interchangeable and I will get same results for : cl <-…
Tomas Greif
  • 21,685
  • 23
  • 106
  • 155
18
votes
2 answers

doParallel "foreach" inconsistently inherits objects from parent environment: "Error in { : task 1 failed - "could not find function..."

I have a problem with foreach that I just can't figure out. The following code fails on two Windows computers I've tried, but succeeds on three Linux computers, all running the same versions of R and…
sssheridan
  • 690
  • 1
  • 6
  • 15
14
votes
3 answers

How can I speed up the training of my random forest?

I'm trying to train several random forests (for regression) to have them compete and see which feature selection and which parameters give the best model. However the trainings seem to take an insane amount of time, and I'm wondering if I'm doing…
François M.
  • 4,027
  • 11
  • 30
  • 81
11
votes
2 answers

How to export multiple function or packages in foreach loop in "R"

I'm trying to decrease run time of my code by using doParallel package in R. I'm calling a function awareRateSIR that some extra packages are used in the body of this function. I get some error like could not find function "vcount" and.. I know…
ElhamMotamedi
  • 199
  • 1
  • 2
  • 12
10
votes
3 answers

how to use %dopar% when only import foreach in DESCRIPTION of a package

How to avoid "could not find function "%dopar%"" in a function of a package when only imports (not depends) foreach in DESCRIPTION of a package? is there a way like foreach::%dopar% as I use foreach::foreach in function? Thank you. Code like: In…
Zhilong Jia
  • 2,329
  • 1
  • 22
  • 34
9
votes
1 answer

Run RSelenium in parallel

How would i go about running RSelenium in parallel. The following is an example using rvest in parallel library(RSelenium) library(rvest) library(magrittr) library(foreach) library(doParallel) URLsPar <- c("http://www.example.com/",…
dimitris_ps
  • 5,849
  • 3
  • 29
  • 55
8
votes
4 answers

Running foreach without returning any value in R

I have a function doSomething() which runs in a foreach loop and as a result saves some calculations as .csv files. Hence I have no need for a return value of foreach, in fact I don't want a return value because it clutters my memory to the point…
elmo
  • 325
  • 1
  • 12
8
votes
1 answer

Parallelizing keras models in R using doParallel

I'm trying to ensemble several neural networks using keras for R. In order to do so, I would like to parallelize the training of the different networks by using a "foreach" loop. models <- list() x_bagged <- list() y_bagged <- list() n_nets =…
Francesco Reggiani
  • 81
  • 2
8
votes
2 answers

loop inside a foreach loop using doparallel

I have a function that contains a loop myfun = function(z1.d, r, rs){ x = z1.d[,r] or.d = order(as.vector(x), decreasing=TRUE)[rs] zz1.d = as.vector(x) r.l = zz1.d[or.d] y=vector() for (i in 1:9) { if(i<9) y[i]=mean( x[(x[,r] >=…
sbg
  • 1,772
  • 8
  • 27
  • 45
8
votes
1 answer

R: show error and warning messages in foreach %dopar%

I'm new to using foreach() %dopar% for paralleling, and I have some problems about how it handles errors or warnings. when I use try() with my customized error message within foreach() %dopar%, the "native" error message doesn't show up: test <-…
Kenneth Cheng
  • 103
  • 1
  • 3
7
votes
2 answers

Caret doparallel in Rmarkdown: Missing verbose info when using render()

I have the following simple example Rmarkdown document (test.Rmd): --- title: "Test Knit Caret Paralell VerboseIter" output: html_document --- ```{r setup, include=FALSE} knitr::opts_chunk$set(echo =…
Wietze314
  • 5,942
  • 2
  • 21
  • 40
7
votes
0 answers

R : Render_site is not working with Dopar

I have a problem with Render_site using the dopar of foreach The code works perfectly using simple for. Whereas I have a problem when using dopar since the ouput is duplicated after using dopar in the YAML file. Note that the dopar works correctly…
Geek
  • 189
  • 3
  • 13
1
2 3
…
30 31