Parallel processing in R using 'doParallel' package

Question

I have a loop that iterates from 2 to a specified value(i.e. ,columnCount). The value of i is crucial as all computation that takes place inside the loop is dependent on the value of i.

Loop snippet:

   > x1=runif(900000,9999,90999)
   > x2=runif(900000,0,9)
   > x3=runif(900000,5000,80000)
   > y=rep(0:1,450000)
   > data=data.frame(y,x1,x2,x3)
   > dim(data)
    [1] 900000   4 
   > columnCount = ncol(data)
   > yVar = names(data[1])

for (i in 2:columnCount) {  
    xVar[i] = names(data[i])
    result <- smbinning(df=data,y=yVar,x=xVar[i],p=0.05)
    }

Note: Y column is always constant,while x columns iterates by 1 in every step(Actual data frame has 250+ columns).How do i translate this to so that I can use:

library(foreach)
library(doParallel)

foreach(icount(iters)) %dopar% {

Please state a full representative and [reproducible problem](http://stackoverflow.com/a/5963610/1412059). The `...` are not irrelevant. Most likely you shouldn't use a loop at all. — Roland, Nov 27 '15 at 17:16
There are more than 100 lines of code there. The loop bins one variable of a data frame at a time using smbinning. — steven, Nov 27 '15 at 17:41
It's your task to create a minimal and reproducible example. We don't want a huge amount of code, but something small that demonstrates your problem and can be run in an R session. — Roland, Nov 27 '15 at 17:44
Please note that the current dataset does not produces bins, the actual dataset takes about 40s to bin one column. — steven, Nov 29 '15 at 14:42

Raad · Accepted Answer · 2015-12-11T17:25:52.233

How about using the mclapply from the parallel package. Something like the below for example:

require(smbinning) # caveat: I never used this package
data(iris) 

names(iris) <- gsub("\\.","",names(iris)) # didn't like dots
mclapply(2:NCOL(iris), function(varb) 
smbinning(df = iris[,c(1,varb)], 
            y = names(iris[,c(1,varb)])[1], 
            x = names(iris[,c(1,varb)])[2],
            p = 0.05))

Parallel processing in R using 'doParallel' package

1 Answers1