1

I am running doMPI on an HPC and I would like to log output from workers. Using doParallel, I was able to use makeCluster(outfile='myfile.log'). With doMPI, there does not seem to be an outfile argument in any of the methods. I tried using sinkWorkerOutput(). This works, but only wrote the log for one of the workers. I suspect that each worker is overwriting the other. Is there an analog for outfile for doMPI?

A related question - inside of a worker, can I find the worker number?

EDIT: here is a link to an answer discussing how to use outfile: How can I print when using %dopar%

Thank you for your help,

Ben

Community
  • 1
  • 1
Ben Carlson
  • 1,053
  • 2
  • 10
  • 18

1 Answers1

0

To send worker output to a file in the doMPI package, set the startMPIcluster "verbose" option to TRUE:

cl <- startMPIcluster(verbose=TRUE)

This creates one file per worker with names of the form "MPI_1_steve_41747.log". The MPI rank, user name, and process ID is used to make the file names unique. You can also specify the log directory via the "logdir" option.

To get a worker number, you can simply call the mpi.comm.rank function.

Steve Weston
  • 19,197
  • 4
  • 59
  • 75