Can I import variables into R from a global file?

Question

I am integrating an R script to produce some graphics into a larger project that is pulled together with a Makefile. In this larger project, I have a file called globals.mk that contains global variables used by many other scripts in the project. For example, the number of simulations I want to run is a global that I want to use in this R script. Can I "import" this as a variable, or is it necessary to manually define every variable within the R script?

Edit: here is a sample of the globals that I would need to read in.

num = 100
path = ./here/is/a/path
file = $(path)/file.csv

And I would like the R script to set the variables num as 100 (or "100"), path as "./here/is/a/path" and file as "./here/is/a/path/file.csv".

You definitely can, but how you'd do it depends on the format of the `globals.mk` file. The easiest way is to [source an R file](https://stackoverflow.com/questions/11051187/reading-and-using-a-custom-configuration-file), but there are options for other formats like [YAML](https://stackoverflow.com/questions/5272846/how-to-get-parameters-from-config-file-in-r-script?noredirect=1&lq=1) and others. Please post an example of your file, or specify what (if any) constraints are placed on that file — divibisan, Jun 25 '19 at 17:54
If global.mk contains lines like `num = 100` (which is valid R source code) then in your R program use `source("global.mk")` to read it in and run it. If it is not valid R source then use `readLines` to read it in and do whatever processing you need to extract and set the values from the character vector that `readLines` returns. Note that questions should include a complete minimal reproducible example. — G. Grothendieck, Jun 25 '19 at 17:56
Thanks, I've added more details and clarified the requirements. — karthikt, Jun 25 '19 at 18:07
You can pass command line parameters when you call an R script. Right now it's unclear exactly how you are calling the script. Please try to provide a [reproducible example](https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) that we can actually run and test with. — MrFlick, Jun 25 '19 at 18:10

r2evans · Answer 1 · 2019-06-26T15:55:29.910

If the .mk file has anything other than direct variable expansion (such as more complex make-rules/tricks/functions), it might be better to trust make to do the expansion for you, and then read it in. There's a post here that I found that dumps all variable contents (after processing).

TL;DR

expand_mkvars <- function(path, aslist = FALSE) {
  stopifnot(file.exists(mk <- Sys.which("make")))
  tf <- tempfile(fileext = ".mk")
  # needed on my windows system
  tf <- normalizePath(tf, winslash = "/", mustWork = FALSE) # tempfile should suffice
  on.exit(suppressWarnings(file.remove(tf)), add = TRUE)
  writeLines(c(".PHONY: printvars",
               "printvars:",
               "\t@$(foreach V,$(sort $(.VARIABLES)), \\",
               "\t   $(if $(filter-out environment% default automatic, \\",
               "\t   $(origin $V)),$(warning $V=$($V))))"), con = tf)
  out <- system2(mk, c("-f", shQuote(path), "-f", shQuote(tf), "-n", "printvars"),
                 stdout = TRUE, stderr = TRUE)
  out <- out[grepl(paste0("^", tf), out)]
  out <- gsub(paste0("^", tf, ":[0-9]+:\\s*"), "", out)
  known_noneed <- c(".DEFAULT_GOAL", "CURDIR", "GNUMAKEFLAGS", "MAKEFILE_LIST", "MAKEFLAGS")
  out <- out[!grepl(paste0("^(", paste(known_noneed, collapse = "|"), ")="), out)]
  if (aslist) {
    spl <- strsplit(out, "=")
    nms <- sapply(spl, `[[`, 1)
    rest <- lapply(spl, function(a) paste(a[-1], collapse = "="))
    setNames(rest, nms)
  } else out
}

In action:

expand_mkvars("~/StackOverflow/karthikt.mk")
# [1] "file=./here/is/a/path/file.csv" "num=100"                       
# [3] "path=./here/is/a/path"         

expand_mkvars("~/StackOverflow/karthikt.mk", aslist = TRUE)
# $file
# [1] "./here/is/a/path/file.csv"
# $num
# [1] "100"
# $path
# [1] "./here/is/a/path"

I have not tested on other systems, so you might need to adjust known_noneed to add extra variables that popup. Depending on your needs, you might be able to filter more-intelligently (e.g., none of your variables lead with a capital letter), but for this example I kept it to the known-not-wanted variables that make is giving us.

The blog post suggests using a phony target of

.PHONY: printvars
printvars:
    @$(foreach V,$(sort $(.VARIABLES)), \
       $(if $(filter-out environment% default automatic, \
       $(origin $V)),$(warning $V=$($V))))

(some are tabs, not all spaces, very important for make)

Unfortunately, it produces more output than you technically need:

$ /c/Rtools/bin/make.exe -f ~/StackOverflow/karthikt.mk printvars
C:/Users/r2/StackOverflow/karthikt.mk:10: .DEFAULT_GOAL=all
C:/Users/r2/StackOverflow/karthikt.mk:10: CURDIR=/Users/r2/Projects/Ford/shiny/shinyobjects/inst
C:/Users/r2/StackOverflow/karthikt.mk:10: GNUMAKEFLAGS=
C:/Users/r2/StackOverflow/karthikt.mk:10: MAKEFILE_LIST= C:/Users/r2/StackOverflow/karthikt.mk
C:/Users/r2/StackOverflow/karthikt.mk:10: MAKEFLAGS=
C:/Users/r2/StackOverflow/karthikt.mk:10: SHELL=sh
C:/Users/r2/StackOverflow/karthikt.mk:10: file=./here/is/a/path/file.csv
C:/Users/r2/StackOverflow/karthikt.mk:10: num=100
C:/Users/r2/StackOverflow/karthikt.mk:10: path=./here/is/a/path
make: Nothing to be done for 'printvars'.

so we need a little filtering, ergo the majority of code in the function.

Edit: it the readRenviron-to-envvar is the best way for you, it would not be difficult to redirect the output of this make call to another file, parse out the relevant lines, and then do readRenviron on that new file. It seems more indirect due to the use of two temp files, but they're cleaned up so that should be nothing to worry about.

Also hinted at here: https://stackoverflow.com/a/7119460 (and other answers there) — r2evans, Jun 25 '19 at 20:06

G. Grothendieck · Accepted Answer · 2019-06-25T21:12:25.033

If it is ok to replace the parentheses with brace brackets then readRenviron will read in such files and perform the substitutions returning the contents as environmental variables.

# write out test file globals2.mk which uses brace brackets
Lines <- "num = 100
path = ./here/is/a/path
file = ${path}/file.csv"
cat(Lines, file = "globals2.mk")

readRenviron("globals2.mk")
Sys.getenv("num")
## [1] "100"
Sys.getenv("path")
## [1] "./here/is/a/path"
Sys.getenv("file")
## [1] "./here/is/a/path/file.csv"

If it is important to use parentheses rather than brace brackets, read in globals.mk, replace the parentheses with brace brackets and then write the file out again.

# write out test file - this one uses parentheses as in question
Lines <- "num = 100
path = ./here/is/a/path
file = $(path)/file.csv"
cat(Lines, file = "globals.mk")

# read globals.mk, perform () to {} substitutions, write out and then re-read
tmp <- tempfile()
L <- readLines("globals.mk")
cat(paste(chartr("()", "{}", L), collapse = "\n"), file = tmp)
readRenviron(tmp)

This is perfect! Sys.getenv() allows me to call the variable, which is what I needed. — karthikt, Jun 26 '19 at 06:52

Can I import variables into R from a global file?

2 Answers2

TL;DR