6

I noticed that R doesn't use all of my CPU, and I want to increase that tremendously (upwards to 100%). I don't want it to just parallelize a few functions; I want R to use more of my CPU resources. I am trying to run a pure IP set packing program using the lp() function. Currently, I run windows and I have 4 cores on my computer.

I have tried to experiment with snow, doParallel, and foreach (though I do not know what I am doing with them really).

In my code I have this...

library(foreach)
library(doParallel)
library(snowfall)

cl <- makeCluster(4)
registerDoParallel(cl)

sfInit(parallel = TRUE, cpus = 4)


#code that is taking a while to run but does not involve simulations/iterations

lp (......, all.int = TRUE)

sfStop()

R gets stuck and runs lp() for a very long time. My CPU is around 25%, but how can I increase that?

Johnson
  • 69
  • 1
  • 3
  • 1
    If you need to run 4 integer programs in parallel, then the packages you are using would be useful. If you need to run a single integer program using all 4 cores, you need to find a package that does that for you (and I don't think the `lpSolve` package provides that option). – josliber Aug 09 '15 at 03:57
  • fwiw, this sounds quite similar to [this recent question](http://stackoverflow.com/questions/31868370/parallel-multicore-processing-in-r-for-an-integer-program); you may find some of the suggested packages in the comments helpful. – josliber Aug 09 '15 at 03:59
  • 1
    Is there a package that allows me to run programs using all my cores? It doesn't necessarily have to be specific to IP, but more so can I increase my R's CPU usage? – Johnson Aug 09 '15 at 04:55
  • Not to my knowledge -- parallelizing functions often requires a pretty deep understanding of how they work. For instance, parallelizing MIPs is quite intricate and an area of active research in operations research and related communities. You can read more about the many R packages for parallel processing [here](https://cran.r-project.org/web/views/HighPerformanceComputing.html). – josliber Aug 09 '15 at 05:11
  • Easiest solution is to use Revolution R's distribution. It uses the CPU's SIMD instructions to achieve 4x speedup just on a single core. It also uses multiple cores a lot more efficiently than R itself. Running `svd` on i7 was 7x times faster on RRO than plain R. Parallelization solutions should be tried *after* you ensure R itself is running at its maximum. – Panagiotis Kanavos Nov 30 '15 at 11:28

2 Answers2

0

If you are trying to run 4 different LPs in parallel, here's how to do it in snowfall.

sfInit(parallel=TRUE, cpus=4)
sfSource(code.R) #if you have your function in a separate file
sfExport(list=c("variable1","variable2",
            "functionname1")) #export your variables and function to cluster
results<-sfClusterApplyLB(parameters, functionname) #this starts the function on the clusters

E.g. The function in the sfClusterApply could contain your LP.

Otherwise see comments in regard to your question

Buggy
  • 2,050
  • 21
  • 27
0

Posting this as an answer because there's not enough space in a comment.
This is not an answer directly towards your question but more to the performance.

R uses slow statistical libraries by default which also can only use single core by default. Improved libraries are OPENBLAS/ATLAS. These however, can be a pain to install.
Personally I eventually got it working using this guide.

I ended up using Revolution R open(RRO) + MKL which has both improved BLAS libraries and multi-cpu support. It is an alternative R distribution which is supposed to have up to 20x the speed of regular R (I cannot confirm this, but it is alot faster).

Furthermore, you could check the CRAN HPC packages to see if there is any improved packages which support the lp function.

There is also packages to explore multi cpu usage.
This answer by Gavin, as well as @user3293236's answer above show several possibilities for packages allowing multi CPU usage.

Community
  • 1
  • 1
Bas
  • 1,066
  • 1
  • 10
  • 28