Rcpp: how to keep files generated by sourceCpp?

Question

I use sourceCpp() from the Rcpp package to build a C++ file and call it from R. It seems to generate a temporary directory where it writes the source it compiles, but it removes that directory after building the code. I want to get access to the exact file it is compiling, so that I can see it in my debugger. How can I prevent sourceCpp() from deleting the file it compiles?

Answer 1

I also love the flexibility of sourceCpp when doing academic researches. In many situations, writing a package is too much for our purposes. I have been using the following wrapper of sourceCpp which keeps the shared library.

importCpp <- function(infile, output_dir="lib", rebuild=FALSE){
    output_dir = ifelse(is.null(output_dir), ".", output_dir)
    dir.create(output_dir, recursive=T, showWarnings=FALSE)
    outfile = file.path(output_dir, paste0(infile, ".R"))

    if (!file.exists(outfile) || file.info(infile)$mtime > file.info(outfile)$mtime || rebuild){
        Rcpp::sourceCpp(infile, rebuild=rebuild)
        context = .Call("sourceCppContext", PACKAGE = "Rcpp",
            normalizePath(infile, winslash = "/"), code=NULL, 0L, .Platform)
        scriptfile = file.path(context$buildDirectory, context$rSourceFilename)
        content = readLines(scriptfile)
        ext = .Platform$dynlib.ext
        m = regexpr(paste0("(?<=dyn.load\\(').*", ext), content[1], perl=TRUE)
        shlibfile = file.path(output_dir, paste0(infile, ext))
        shlibfile0 = regmatches(content[1], m)
        content[1] = sub(shlibfile0, shlibfile, content[1])

        f = file(outfile, "w+")
        writeLines(content, f)
        close(f)
        file.copy(shlibfile0, shlibfile, overwrite=TRUE)
    }else{
        source(outfile)
    }
    invisible(outfile)
}

To use the code:

importCpp("foo.cpp")

If the file has not beed complied, the shared library and a R file foo.cpp.R will be copied to a folder lib under the current folder. However, if lib\foo.cpp.R is found, it would source the R file.

Also, if foo.cpp is modified, importCpp will recompile the cpp file if necessary.

Answer 2

As Dirk said -- seriously, just use packages. Any other workflow you develop on top of sourceCpp is likely to be insufficient / annoying to use.

That said, Rcpp does not automatically delete the directory where source files are generated. Look at the output from sourceCpp(..., verbose = TRUE):

> sourceCpp("~/scratch/save-source-cpp.cpp", verbose = TRUE)

Generated extern "C" functions 
--------------------------------------------------------


#include <Rcpp.h>

RcppExport SEXP sourceCpp_2047_timesTwo(SEXP xSEXP) {
BEGIN_RCPP
    SEXP __sexp_result;
    {
        Rcpp::RNGScope __rngScope;
        Rcpp::traits::input_parameter< int >::type x(xSEXP );
        int __result = timesTwo(x);
        PROTECT(__sexp_result = Rcpp::wrap(__result));
    }
    UNPROTECT(1);
    return __sexp_result;
END_RCPP
}

Generated R functions 
-------------------------------------------------------

`.sourceCpp_2047_DLLInfo` <- dyn.load('/var/folders/tm/5dt8p5s50x58br1k6wpqnwx00000gn/T//RtmppARq3j/sourcecpp_1b612a59c474/sourceCpp_85891.so')

timesTwo <- Rcpp:::sourceCppFunction(function(x) {}, FALSE, `.sourceCpp_2047_DLLInfo`, 'sourceCpp_2047_timesTwo')

rm(`.sourceCpp_2047_DLLInfo`)

Building shared library
--------------------------------------------------------

DIR: /var/folders/tm/5dt8p5s50x58br1k6wpqnwx00000gn/T//RtmppARq3j/sourcecpp_1b612a59c474

/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB -o 'sourceCpp_85891.so' 'save-source-cpp.cpp' 
clang++ -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include  -I"/Users/kevin/Library/R/3.1/library/Rcpp/include"    -fPIC  -g -O3 -Wall -pedantic -c save-source-cpp.cpp -o save-source-cpp.o
clang++ -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/usr/local/lib -o sourceCpp_85891.so save-source-cpp.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation

Note the DIR: line. If I look at what's within, I see:

> list.files("/var/folders/tm/5dt8p5s50x58br1k6wpqnwx00000gn/T//RtmppARq3j/sourcecpp_1b612a59c474")
[1] "save-source-cpp.cpp"   "save-source-cpp.cpp.R" "save-source-cpp.o"     "sourceCpp_29322.so" 

and if I read the (modified) cpp file, I see:

> cat(readLines("/var/folders/tm/5dt8p5s50x58br1k6wpqnwx00000gn/T//RtmppARq3j/sourcecpp_1b612a59c474/save-source-cpp.cpp"), sep = "\n")
#include <Rcpp.h>
using namespace Rcpp;

// Below is a simple example of exporting a C++ function to R. You can
// source this function into an R session using the Rcpp::sourceCpp
// function (or via the Source button on the editor toolbar)

// For more on using Rcpp click the Help button on the editor toolbar

// [[Rcpp::export]]
int timesTwo(int x) {
   return x * 2;
}


#include <Rcpp.h>

RcppExport SEXP sourceCpp_2047_timesTwo(SEXP xSEXP) {
BEGIN_RCPP
    SEXP __sexp_result;
    {
        Rcpp::RNGScope __rngScope;
        Rcpp::traits::input_parameter< int >::type x(xSEXP );
        int __result = timesTwo(x);
        PROTECT(__sexp_result = Rcpp::wrap(__result));
    }
    UNPROTECT(1);
    return __sexp_result;
END_RCPP
}

So all we do is generate the wrapper function using the .Call interface that can understand the regular function interface.

If you want to save that file somewhere, you could use capture.output(sourceCpp(..., verbose = TRUE)) and then parse the DIR: output and go from there.

But seriously, just make a package.

Answer 3

Reactivating this question since it was not answered and it is still my googles first hit.

I have a C version of an R function saved in the file f1.cpp. What I (and the op) want is the result of sourceCpp("f1.cpp"), namely the file f1.so which is hidden in the temporary folder defined via the argument cacheDir of the function sourceCpp.

If, however, I set cacheDir=/my/path to a director of my choice, still, temporary directories are created and the result is not saved as /my/path/f1.so but as /my/path/sourceCpp-x86_64-pc-linux-gnu-1.0.1/sourcecpp_2db138ae3a0c/sourceCpp_2.so.

This is very cumbersome since now I have to

1. find and mv /my/path/asd/asd/sourceCpp_2.so /my/path/f1.so
2. add the rather complicated line to the R script where the C version of the function finally shall be used:

f1 <- Rcpp:::sourceCppFunction(function(arg1, arg2) {},
                               isVoid=F, dll=dyn.load("my/path/f1.so"),
                               symbol='sourceCpp_1_f1') # from the generated file f1.cpp.R
res <- f1(arg1, arg2)

3. Finally, I can add the well defined /my/path and all its content to the project I am working on (e.g. git*).

I dont see how "make a package" is an answer to this question.

Answer 4

Please see the Rcpp Attributes vignette about how to transition from one-off files for experimentation and exploration (which are compiled via sourceCpp() to organizing your files -- using the same technology -- in the way R prefers: as packages.

It is not hard, Rcpp Attributes helps you, and you will see the advantage in due course. You could start via Rcpp.package.skeleton() (see its help page) or copy from an existing package.

(As an aside, the temporary directory used by sourceCpp() is the same temporary directory used by the very R session you are in when you call sourceCpp(). We don't obfuscate any further.)