"Back engineering" an R package from compiled binary version

Question

I work for an organization with a number of internal R packages, all written many years ago. These are stored as .zip archives built on Windows under R 3.x. They cannot be loaded on Linux or macOS or under R 4.y without being rebuilt. Unfortunately, I do not have access to the package sources. They are lost to time...

I want to take these binaries, extract the source code, and repackage it according to current best practices (version control, roxygen2, testthat, etc.). What is the best way to do that?

I have already tackled one of the binaries by:

1. Manually copying the source code of objects (exported and internal functions, data sets, etc.) in the loaded namespace to new .R files.
2. Manually adding Roxygen blocks to the .R files in order to reproduce the help pages as displayed in the browser.

I am partly stuck at (1) because some of the functions are S4 generic. dput(<name>) gives new("standardGeneric", ...) as opposed to a simple function definition. Otherwise, the process has been fairly straightforward, but very time consuming.

Is there a way to programmatically "back engineer" source files from R package binaries, while handling S4 generic functions, classes, and methods correctly?

Everyone in the organization will be stuck on R 3.6 until this problem is resolved.

Answer 1

Check out if this works in R 3.6.

Below script can automate least part of your problem by writing all function sources into separate and appropriately named .R files. This code will also take care of hidden functions.

Extracting code

# Use your package name
package_name <- "dplyr" 

# Extract all method names, including hidden
nms <- paste(lsf.str(paste0("package:", package_name), all.names = TRUE))

# Loop through the method names,
# extract head and body, and write them to R files
for (i in 1:length(nms)) {

    # Extract name
    nm <- nms[i]

    # Extract head
    hd_raw <- capture.output(args(nms[i]))
    # Collapse raw output, but drop trailing NULL
    hd <- paste0(hd_raw[-length(hd_raw)], collapse = "\n")

    # Extract body, collapse
    bd <- paste0(capture.output(body(nms[i])), collapse = "\n")
    
    # Write all to file
    write(paste0(hd, bd), file = paste0(nm, ".R"))
}

Extracting help files

To extract a functions's help text a similar way, you can use code from the following SO answers:

• for plain text: Get the documentation of an R function from the help as a string
• for .Rd file contents: How to access the help/documentation .rd source files in R?

A starting point could be something like:

library(tools)
package_name <- "dplyr" 
db <- Rd_db(package_name)

# Extract all method names, including hidden
nms <- paste(lsf.str(paste0("package:", package_name), all.names = TRUE))

# Loop through the method names,
# extract Rd contents if they exist in this namespace, 
# and write them to new Rd files
for (i in 1:length(nms)) {
    
    # Extract name
    nm <- nms[i]
    
    rd_raw <- db[names(db) %in% paste0(nm, ".Rd")]
    if (length(rd_raw) > 0) {
        rd <- paste0(capture.output(rd_raw), collapse = "\n")
        # Write all to file
        write(rd, file = paste0(nm, ".Rd"))
    }
    
}