10

I want to monitor some pretty lengthy parallelized computations embedded in a knitr file.

The computations rely on a package I have written, and the relevant function uses mclapply from the multicore package for parallelization. This function outputs progress bars to monitor the progress of the computations using a slightly modified implementation of txtProgressBar from the utils package. The progress bar is printed to the terminal and updated through a fifo connection every time an iteration of mclapply is complete.

This works fine when sourcing from a file or calling directly the function, but I find no way to get this to work within knitr. I have tried the relevant chunk options, I can get messages and warnings redirected to the terminal, but not the progress bar. Can anyone help?

Sorry for not providing a minimal working example but I don't see how I could make one in this setting.

Pepin_the_sleepy
  • 297
  • 1
  • 13

2 Answers2

8

Because txtProgressBar() writes to stdout, and knitr captures everything in stdout, so currently it is not easy to show your progress bar if it is text-based and writes to stdout. Perhaps I can use evaluate::evaluate(debug = TRUE) internally to achieve what you want, but I'm not entirely sure if that works well with the text progress bar.

My suggestions at the moment are:

  • use a GUI-based progress bar like tcltk::tkProgressBar()

  • write the progress to other places, e.g. (ab)using stderr

    ```{r progress}
pb = txtProgressBar(min = 0, max = 100, file = stderr())
for (i in 1:100) {
  setTxtProgressBar(pb, i)
  Sys.sleep(0.05)
}
close(pb)
```

  • or use your function outside a code chunk, e.g. in an inline expression (such as \Sexpr{my_multicore_function()} in Rnw or `r my_cool_fun()` in Rmd), because inline evaluation does not capture stdout

Yihui Xie
  • 28,913
  • 23
  • 193
  • 419
  • Thanks for the reply @Yihui, I was hoping you'd drop by! I will try something along the line of your first suggestion, the second one would imply loosing the capacity to cache the computations. Multicore issues aside, are you aware of any way to monitor the status of computations conducted within a chunk? – Pepin_the_sleepy Nov 17 '13 at 09:47
  • 1
    Perhaps you can write the progress to other places, such as in a file: `txtProgressBar(..., file = 'progress_temp.txt')`. Or write it to stderr. I'll update my answer. – Yihui Xie Nov 18 '13 at 03:27
  • thanks for the `StdErr()` hack, simple to implement and works wonders. While I am at it, thanks for knitr it's an awesome tool that I use daily. – Pepin_the_sleepy Nov 20 '13 at 17:16
  • @Yihui Is there a way to get knitr to not capture stdout? I want to run knitr docs, but when things go wrong, I don't see any of the debug messages, warnings, .. – dfrankow Jan 20 '17 at 19:21
  • @YihuiXie "knitr captures everything in stdout" As fas I see this is true with `print(1+1)` but not for symply `1+1`. In this case there are not any output in the HTML or the PDF. Of course , there are output if a run `1+1` in a python console (2.7). This is the normal behaviour of knitr or I am doing something wrong? – Fran Jan 06 '19 at 18:50
1

Having read the point about printing progress bar to stderr in Yihui's answer I would suggest temporary redirecting stdout to stderr with sink().

sink(stderr())

your_code()

sink()
konstunn
  • 355
  • 4
  • 17
  • 1
    This is helpful when you don't have control over where your progress bar is sent (such as if using someone else's package like `rstan`) – Empiromancer Apr 04 '18 at 17:26