0

My understanding was that if registering an implicit cluster with e.g. registerDoParallel(cores = 14) then at the end of the loop the cluster is automatically closed.

In trying to debug another issue I wanted to see if the links to the workers (I don't know the correct term) where indeed closed.

Onther doMC documentation under "Getting information about the parallel backend": "To find out how many workers foreach is going to use, you can use the getDoParWorkers function: > getDoParWorkers()"

I expected that after running a dopar loop this would go back to default. However, I notice that it's being remembered since if I currently run getDoParWorkers() it returns 14 which is what I declared in my code block with registerDoParallel(cores = 14) However if I do registerDoParallel(cores = 10) then do a dopar loop, then getDoParWorkers() returns 10.

Does this mean there is still a registered parallel back end even though I completed the loop?

Doug Fir
  • 19,971
  • 47
  • 169
  • 299
  • Hey @Doug: I'm still learning about all the details of parallel myself, but yes I'm pretty sure your backend is still live. I explicitly open and close parallel backends with `cl <- makeCluster(N)` and `stopCluster(cl)`, respectively. Also note that `foreach` does not always use all of your workers; if you're on a Linux, you can test this by running `top` (from Linux command) while your R script runs in the background. `foreach` *can* use up to *N* workers. – CPak Aug 26 '17 at 13:43
  • HI @ChiPak thanks for commenting. One of the benefits of doparallel that I like is that ```registerDoParallel()``` function implicitly creates the cores for you and you don't have to worry about exporting functions and objects as you do with ```makeCluster()``` (because it uses fork?). I might go back to try manually creating the cluster per your suggestion, but do you know if it's possible to force close the backend after a dopar loop? I tried ```stopImplicitCluster()``` but then after that ```getDoParWorkers()``` still shows 14 – Doug Fir Aug 26 '17 at 17:36
  • Can you just use `registerDoParallel(cores = 1)` after the loop has completed I wonder? – Seanosapien Dec 14 '19 at 20:47

0 Answers0