4

I have an RMarkdown report that is very useful and has grown to be several pages long with all the figures and tables in the HTML file.

It uses the same dataset for all the figures and tables.

What I would like to do is to keep generating this large html file and then several new subdirectories, each with their own html files and subdirectories within those, each with their own html files.

In this case, the full report contains data on a department, then each subdirectory would contain an html output related to each group within the department, and each of those would contain a subdirectory with html output for each person in each group. This way if someone is only interested in the metrics of one group, or one person, they look at the most appropriate output.

Parent dir: The same large html file with figures and tables generated with data for entire dept.
|
 __Subdir for each group: Output based on same data but only the group's metrics
    |
     __Subdir for each person: Output based on same data but only individual's metrics

What's the best way to arrange this?
1. Is there a code chunk option in RMardkown where I can say, chunk a goes in this html output file, chuck b goes in another?
2. Do I need multiple RMarkdown files, one for each html output, witch some sort of caching between them so I don't have to reprocess all the data? (this would seem silly because I need a lot of html files)
3. Should I give up RMarkdown for this task?

variable
  • 1,013
  • 1
  • 12
  • 29
  • 1
    There are ways to programmatically create markdown documents with knitr. If you had a markdown file in each subdirectory you could walk the tree and create a master document of all the individual pieces. If I get time later I will try and make a proper answer for you. – Ryan Hope Oct 10 '14 at 20:11
  • I'm wondering if there's any way to avoid loading the data for every Rmd file too. – variable Oct 20 '14 at 15:36

1 Answers1

3

I do something like you're proposing with knitr, and it works very well.

Don't tell anyone, but I use a 'for' loop to cycle through a bunch of councils, each of whom get the same report but with their data. I then push the report into a directory structure, zip it and mail it.

I have an Rmd file that expects two datasets, setA (being the subject) and setB (being its peers)

The flow is something like:

set <- assemble_data() # loads whole set
for (report in report_list) {
    setA <- filter(set, subject == report)
    setB <- filter(set, subject != report)
    output_html <- str_c('path/',report,'.html')
    knit_interim <- str_c('path/',report,'md')
    knit_pattern <- 'name of RMd' # I generate more than one report for each place
    knit(knit_pattern) 
    markdowntoHTML(file = knit_interim, output=output_html, stylesheet=stylesheet, encoding='windows-1252')
}

In this way I can produce a report set in a few minutes. My case may be simpler than yours, because the report structure is the same - it's the datasets that change.

Note that this is not a paste of the code (it is slightly more complicated than this) so beware typos etc.

The point (as I understand it) is to write an Rmd that expects a dataset of a particular name, and the R code provides local scope for it. I struggled initially with it but it's all quite simple in its execution.

[update: 'How do you pass the data to the RMd files ?'

You don't explicitly need to. In my code above the RMd is written expecting data in setA and setB.

It makes the workflow really easy - you write the template using a dataset (manually filter for one) and then when you're ready you can just run the loop. Like I said, I struggled a bit to understand at first but just jumped in and it all worked out quite nicely.

Richard Crawley
  • 98
  • 1
  • 8
  • Hi, how do you pass the data to the md files? – variable Oct 19 '14 at 23:54
  • 1
    The trick is to create a template for the content you want to generate. https://github.com/yihui/knitr-examples/blob/master/041-label-i.Rmd – Ryan Hope Oct 20 '14 at 18:29
  • Here is a working SO example that implements the template concept mentioned by Ryan Hope/Yihui Xie: http://stackoverflow.com/questions/30422008/r-knitr-pdf-is-there-a-posssibility-to-automatically-save-pdf-reports-generate/30512430#30512430 – Stan Dec 22 '15 at 14:38
  • I am finding that `rmarkdown::render` works better, preserving markdown such as ``## Subheading`` and floating table of contents. – Frank Harrell Oct 21 '18 at 16:35
  • @FrankHarrell: I am keen to know if your answer that `render` is faster / better than looping inside the `.Rmd` and outputting html files using `markdowntoHTML` (as suggested by Richard Crawley) will still hold good if the data loading from a disk file happens on _every_ `render()`. Assume we have to loop this a hundred times with 100 html files to be written out. – Lazarus Thurston Sep 25 '19 at 14:59
  • BTW Richard Crawley i didn't find the function `markdowntoHTML()` in `rmarkdown` v1.9. – Lazarus Thurston Sep 25 '19 at 15:07