Namespaces without packages

Question

In reorganising my code base I’d like to clean up my code sharing mechanism. So far I’m using source for lots of small, largely self-contained modules of functionality.

However, this approach suffers from a number of problems, among them

• the lack of tests for circularity (accidental circular source chains),
• complex syntax required to properly specify include paths (chdir=TRUE argument, hard-coded paths),
• potential of name clashes (when redefining objects).

Ideally I’d like to get something alike to the Python module mechanism. The R package mechanism would be overkill here: I do not want to generate nested path hierarchies, multiple files with tons of metadata and manually build the package just to get a small, self-contained, reusable code module.

For now I’m using a code snippet which allows me to solve the first two problems mentioned above. The syntax for inclusion is like this:

import(functional)
import(io)
import(strings)

… and a module is defined as a simple source file which resides in the local path. The definition of import is straightforward but I cannot solve the third point: I want to import the module into a separate namespace but from what I see the namespace lookup mechanism is pretty hard-wired to packages. True, I could override `::` or getExportedValue and maybe asNamespace and isNamespace but that feels very dirty and has the potential of breaking other packages.

Answer 1

Here's a function that completely automates package creation, compilation, and reloading. As others have noted, the utility functions package.skeleton() and devtools::load_all() already get you almost all the way there. This just combines their functionality, using package.skeleton() to create the source directory in a temp directory that gets cleaned up when load_all() is done processing it.

All you need to do is point to the source files from which you want to read in functions, and give the package a name: import() does the rest for you.

import <- function(srcFiles, pkgName) {
    require(devtools)
    dd <- tempdir()
    on.exit(unlink(file.path(dd, pkgName), recursive=TRUE))
    package.skeleton(name=pkgName, path = dd, code_files=srcFiles)
    load_all(file.path(dd, pkgName))
}

## Create a couple of example source files
cat("bar <- function() {print('Hello World')}", file="bar.R")
cat("baz <- function() {print('Goodbye, cruel world.')}", file="baz.R")

## Try it out
import(srcFiles=c("bar.R", "baz.R"), pkgName="foo")

## Check that it worked
head(search())
# [1] ".GlobalEnv"        "package:foo"       "package:devtools"
# [4] "package:stats"     "package:graphics"  "package:grDevices"
bar()
# [1] "Hello World"
foo::baz()
# [1] "Goodbye, cruel world."

Answer 2

Konrad, in all seriousness, the answer to the demand

to get a small, self-contained, reusable code module

is to create a package. That gospel has been repeated numerous times here on SO, and in other places. You can in fact create minimal packages with minimal fuzz.

Also, after running

 setwd("/tmp")
 package.skeleton("konrad")

and removing the one temporary file, I am left with

 edd@max:/tmp$ tree konrad/
 konrad/
 ├── DESCRIPTION
 ├── man
 │   └── konrad-package.Rd
 └── NAMESPACE

 1 directory, 3 files
 edd@max:/tmp$ 

Is that really that onerous?

Answer 3

A package is just a convention for where to store files (R files in R/, docs in man/, compiled code in src, data in data/): if you have more than a handful of files, you're best sticking with established convention. In other words, using a package is easier than not using a package, because you don't need to think: you can just take advantage of existing conventions and every R user will understand what's going on.

All a minimal package really needs is a DESCRIPTION file, which says what the package does, who can use it (the license), and who to contact if there are problems (the maintainer). This is a bit of an overhead, but it's not major. Once you've written that, you just fill in the additional directories as you need them - no need for the clumsy package.skeleton().

That said, the built-in tools for working with packages are cumbersome - you have to re-build/re-install the package, restart R and reload the package. That's where devtools::load_all() and Rstudio's build & reload come in - they use the same specification for a package, but provide easier ways to update a package from source. You can of course use the code snippets provided by the other answers, but why not use well tested code that's used by hundreds (well, tens at least) of R developers?

Answer 4

My comment to the OP's question wasn't quite right, but I think this re-write of the import function does the trick. foo.R and bar.R are files in the current working directory that contain a single function (baz) that prints the output shown below.

import <- function (module) {
  module <- as.character(substitute(module))
  # Search path handling omitted for simplicity.
  filename <- paste(module, 'R', sep = '.')
  # create imports environment if it doesn't exist
  if ("imports" %in% search())
    imports <- as.environment(match("imports",search()))
  # otherwise get the imports environment
  else
    imports <- attach(NULL, name="imports")
  if (module %in% ls("imports"))
    return()
  # create a new environment (imports as parent)
  env <- new.env(parent=imports)
  # source file into env
  sys.source(filename, env)
  # ...and assign env to imports as "module name"
  assign(module, env, imports)
}
setwd(".")
import(foo)
import(bar)
foo$baz()
# [1] "Hello World"
bar$baz()
# [1] "Buh Bye"

Note that baz() by itself won't be found, but the OP seemed to want the explicitness of :: anyway.

Answer 5

I'm wholly sympathetic with @Dirk's answer. The small overhead involved in making a minimal package seems worth conforming to a "standard way".

However, one thing that came to mind is source's local argument, letting you source into an environment, which you could use like a namespace, e.g.

assign(module, new.env(parent=baseenv()), envir=topenv())
source(filename, local=get(module, topenv()), chdir = TRUE)

To access these imported environments with a simple syntax, give these import environments an new class (say, 'import'), and make :: generic, defaulting to getExportedValue when pkg doesn't exist.

import <- function (module) {
    module <- as.character(substitute(module))
    # Search path handling omitted for simplicity.
    filename <- paste(module, 'R', sep = '.')

    e <- new.env(parent=baseenv())
    class(e) <- 'import'
    assign(module, e, envir=topenv())
    source(filename, local=get(module, topenv()), chdir = TRUE)
}

'::.import' <- function(env, obj) get(as.character(substitute(obj)), env)
'::' <- function(pkg, name) {
    pkg <- as.character(substitute(pkg))
    name <- as.character(substitute(name))
    if (exists(pkg)) UseMethod('::')
    else getExportedValue(pkg, name)
}

Update

Below is a safer option that would prevent errors in the case that a loaded package contains an object with the same name as a package being accessed with ::.

'::' <- function(pkg, name) {
    pkg.chr <- as.character(substitute(pkg))
    name.chr <- as.character(substitute(name))
    if (exists(pkg.chr)) {
        if (class(pkg) == 'import')
            return(get(name.chr, pkg))
    }
    getExportedValue(pkg.chr, name.chr)
}

This would give the correct result, say, if you loaded data.table, and subsequently tried to access one of its objects with ::.

Answer 6

I’ve implemented a comprehensive solution and published it as a package, ‘box’.

Internally, ‘box’ modules uses an approach similar to packages; that is, it loads the code inside a dedicated namespace environment and then exports selected symbols into a module environment which is returned to the user, and optionally attached. The main difference to packages is that modules are more lightweight and easier to write (each R file is its own module), and can be nested.

Usage of the package is described in detail on its website.