0

I am now dealing with large matrix data and want to accelerate the speed. At first I think parallelization may be a good solution, but then I was told that parallelization for a single call is not possible.

Here is an example,When I try the following code, the speed is not increased as the CPU occupation of the desktop remains around 30%, which is the normal value. So, my question is how to accelerate the calculation speed for just one-line code in R?

set.seed(1234)
mydata=rnorm(5000*150)
mydata=matrix(mydata,ncol=150)

library(parallel)
cl.cores <- detectCores()
cl <- makeCluster(cl.cores)
clusterSetRNGStream(cl,iseed=1234)
clusterExport(cl,"mydata")
clusterEvalQ(cl,library(NbClust))
nc = clusterApply(cl,2,function(min.nc) fun=NbClust(mydata,min.nc=min.nc,max.nc = 8,method = "kmeans"))
stopCluster(cl)
Yang Yang
  • 858
  • 3
  • 26
  • 49
  • 2
    You can not parallelize a single call to a function on all your data. You can parallelize when you have several calls (ie if you can split your data) – HubertL Nov 14 '16 at 23:51
  • So you mean parallelization for a single call is not possible? – Yang Yang Nov 15 '16 at 01:17
  • Exactly, parallelizing means splitting the work to be done on several cpu, which is not possible if you can't split neither the code or the data – HubertL Nov 15 '16 at 01:29
  • Thank you for your help. But why can a parallel loop work in R? Please see http://stackoverflow.com/questions/38318139/run-a-for-loop-in-parallel-in-r – Yang Yang Nov 15 '16 at 02:49
  • 2
    A loop is n calls of a function – HubertL Nov 15 '16 at 03:02
  • Haha, I see it. So, is there any way to accelerate my code? Thanks a lot. – Yang Yang Nov 15 '16 at 03:03
  • googling "cran parallel k-means cluster" gives several options, but I don't know if there's an easy way to put this together with the clustering statistics that `NbClust` gives you ... – Ben Bolker Nov 15 '16 at 03:18
  • Thanks. I know how to do parallel k-means. The problem is how to accelerate `nc = NbClust(mydata,min.nc=2,max.nc = 8,method = "kmeans"))`. – Yang Yang Nov 15 '16 at 03:39

0 Answers0