How can I `print` or `cat` when using parallel

Question

If I call a function using parSapply then print, message or cat statements inside that function don't seem to output to the console.

My process takes a very long time, so I need some way of seeing the progress and getting the results output as they are done. Are there any special commands that would allow me to print to the console from a parallel process?

Example:

library(parallel)

oneloop = function(x) {
  for(i in 1:50) {
    a = rnorm(100000)
    a = sort(a)
  }
  print(x)
  message(x)
  cat(x)
}

cl <- makeCluster(5)
output = parSapply(cl, 1:10, oneloop)
stopCluster(cl)

@TylerRinker, nice idea, but nope, makes no difference. I think the outputs for the worker processes must not be captured... ...so the question might be "can you redirect worker stdout to the main stdout", but I'm speculating now. — Corvus, May 23 '13 at 15:30
Closely related Q&A: http://stackoverflow.com/q/10903787/1036500 — Ben, May 24 '13 at 04:36

eddi · Accepted Answer · 2013-05-23T15:37:01.850

49

Using outfile param in makeCluster you can redirect the output to a file and then check that file to see how your program progresses.

Interestingly on a Linux machine setting it to "" outputs to the console, but that doesn't work for me on a Windows machine. File output works on both.

edited May 23 '13 at 15:37

answered May 23 '13 at 15:31

eddi

49,088
6
104
155

2

A slightly different approach to getting a progress log file when using `%dopar%` is documented in detail here: http://viksalgorithms.blogspot.com/2012/02/monitoring-progress-inside-foreach-loop.html – Ben May 24 '13 at 04:45
`outfile=""` doesn't work on Windows with Rgui, but it does with Rterm. – Steve Weston Oct 12 '13 at 16:13
4

It's strange how poorly-documented this is. I had to pore through source code for a good 15 minutes before I found exactly what `outfile` is doing (basically being added to a command being sent to terminal as `OUT=` `outfile`). Sending to terminal with `""` is consistent with, e.g., `write.table`, which takes the connection `""` to mean `stdout` – MichaelChirico May 03 '16 at 19:49
Using the setting `outfile = ""` to print to the terminal works for me using emacs and ESS in Windows. – JMT2080AD Dec 14 '17 at 19:05

score 4 · Answer 2 · answered Jan 29 '15 at 19:11

4

I am using makeCluster with outfile on Windows with doParallel package version 1.0.8. I'm able to tail outfile using a simple Powershell command powershell Get-Content c:/path/to/log/mylog.txt -wait. Works for me.

answered Jan 29 '15 at 19:11

Daisuke Aramaki

316
3
7

How can I `print` or `cat` when using parallel

2 Answers2

Linked

Related