3

I have seen the question asked here from 2018. I'm wondering if there is a better answer today.

Our work computers are bedeviled by an entire IT security department that seems to exist to make them useless. We are allowed to run R 3.6.3 (4.x hasn't been approved yet). We cannot connect to CRAN from behind the corporate firewall. In the past that meant we took our laptops home to install packages. But now we have a download monitor that blocks CRAN downloads even when we're on our own wi-fi.

I was attempting to get around this by downloading the package .zip files on a personal machine, transferring them via CD, and then installing with repos=NULL. I used this code

getPackages <- function(packs){
  packages <- unlist(
    tools::package_dependencies(packs, available.packages(),
                         which=c("Depends", "Imports"), recursive=TRUE)
  )
  packages <- union(packs, packages)
  packages
}

to get the dependencies for the package I wanted and downloaded them all with download.packages(). I was trying to install tidyquant so it turned out there were 113 dependencies.

But of course this downloads the latest version of all 113 packages, several of which it turned out were incompatible with R 3.6.3.

The solution I referenced above suggested finding a compatible version and then using

install_version("<package name>",
                version = "<version number>",
                repos = "http://cran.us.r-project.org")

But that would involve manually searching all 113 dependencies...plus I'm looking for a general purpose solution that I can share with the ~10 other people in my office that all have this issue now.

I'm hoping that since 2018 a better solution may have presented itself?

Update: Based on answers below I tried

tmp <- tempfile()
dir.create(tmp)

checkpoint::checkpoint(snapshot_date = "2020-04-01", 
                       r_version = "3.6.3", 
                       checkpoint_location = tmp,
                       scan_now = FALSE)


packages <- getPackages(c("tidyquant"))

download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")

I get

Running create_checkpoint in the home directory may result
in checkpointing very many packages. Continue? (Y/n) Y

Creating checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
Using checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
Warning messages:
1: In create_checkpoint(snapshot_date, r_version, checkpoint_location,  :
  Specified R version not the same as current R version
2: In use_checkpoint(snapshot_date, r_version, checkpoint_location,  :
  Specified R version not the same as current R version
> 
> packages <- getPackages(c("tidyquant"))
> 
> download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
  cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant",  :
  no package ‘tidyquant’ at the repositories
Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant",  :
  no package ‘lubridate’ at the repositories
Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant",  :
  no package ‘PerformanceAnalytics’ at the repositories

And the "no package" warnings go on for all 113 dependencies.

I think the key here is

Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:

Obviously there wouldn't be a 4.1 repository in April 2020. I assume it's trying that since the machine is running 4.1. So it seems we're tantalizingly close....

If it helps

> sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22000)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    
system code page: 65001

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.1.2   tools_4.1.2      checkpoint_1.0.2
jerH
  • 1,085
  • 1
  • 12
  • 30
  • you can use conda that's how I had to do. I had to use 4.0 version for some of the libraries. I made separate conda environment for the R version and installed the dependencies – PesKchan Feb 07 '22 at 16:26
  • I haven't really used it, but maybe take a look at the online [RStudio Package Manager](https://packagemanager.rstudio.com/client/#/repos/1/overview). You can fix it to a point in time and download packages as of that date. –  Feb 07 '22 at 16:36

1 Answers1

3

I'm not sure if it completely addresses your needs, but package checkpoint seems appropriate here. It allows you to download source packages from a snapshot of CRAN taken at a specified date, going back to 2014-09-17. R 4.0.0 was released around 2020-04-24, so the snapshot from 2020-04-01 should work for your purposes.

Here is a reproducible example:

tmp1 <- tempfile()
dir.create(tmp1)
cwd <- setwd(tmp1)

getOption("repos")
##                           CRAN 
## "https://cloud.r-project.org/"

available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
##         Version   Repository                               
## lattice "0.20-45" "https://cloud.r-project.org/src/contrib"
## Matrix  "1.4-0"   "https://cloud.r-project.org/src/contrib"
## nlme    "3.1-155" "https://cloud.r-project.org/src/contrib"

download.packages("Matrix", ".", type = "source")
## trying URL 'https://cloud.r-project.org/src/contrib/Matrix_1.4-0.tar.gz'
## Content type 'application/x-gzip' length 2849865 bytes (2.7 MB)
## ==================================================
## downloaded 2.7 MB
## 
##      [,1]     [,2]                   
## [1,] "Matrix" "./Matrix_1.4-0.tar.gz"

tmp2 <- tempfile()
dir.create(tmp2)
checkpoint::checkpoint(snapshot_date = "2020-04-01", 
                       r_version = "3.6.3", 
                       checkpoint_location = tmp2,
                       scan_now = FALSE)
## Creating checkpoint directory /var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T//RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
## Using checkpoint directory /private/var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T/RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
## Warning messages:
## 1: In create_checkpoint(snapshot_date, r_version, checkpoint_location,  :
##   Specified R version not the same as current R version
## 2: In use_checkpoint(snapshot_date, r_version, checkpoint_location,  :
##   Specified R version not the same as current R version

getOption("repos")
##                                             CRAN 
## "https://mran.microsoft.com/snapshot/2020-04-01"

available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
##         Version   Repository                                                  
## lattice "0.20-40" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
## Matrix  "1.2-18"  "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
## nlme    "3.1-145" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"

download.packages("Matrix", ".", type = "source")
## trying URL 'https://mran.microsoft.com/snapshot/2020-04-01/src/contrib/Matrix_1.2-18.tar.gz'
## Content type 'application/octet-stream' length 1871705 bytes (1.8 MB)
## ==================================================
## downloaded 1.8 MB
## 
##      [,1]     [,2]                    
## [1,] "Matrix" "./Matrix_1.2-18.tar.gz"

setwd(cwd)
unlink(c(tmp1, tmp2), recursive = TRUE)

The warnings about version mismatch occur if you are not actually running R 3.6.3. They can be ignored if you are only downloading source packages, with the intention of installing them on another machine actually running 3.6.3.

You can take a look at the package README and ?checkpoint for more details.

Update

If you are trying to download binary packages (.zip for Windows, .tgz for macOS) rather than source packages (.tar.gz), then checkpoint can get you into trouble. By default, download.packages and friends use contrib.url(repos, type) to construct a URL to search for package binaries.

contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "win.binary")
## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1"

contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "mac.binary")
## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1"

But there is nothing at either URL. That is (in part) because contrib.url appends the R version that you are currently running, which might not have existed on your snapshot date. Hence:

download.packages("Matrix", ".", type = "win.binary")
## Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
##   cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
## Warning in download.packages("Matrix", ".", type = "win.binary") :
##   no package 'Matrix' at the repositories
##      [,1] [,2]

download.packages("Matrix", ".", type = "mac.binary")
## Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1:
##   cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1/PACKAGES'
## Warning in download.packages("Matrix", ".", type = "mac.binary") :
##   no package 'Matrix' at the repositories
##      [,1] [,2]

The URLs that you actually need are:

## Windows
"https://cran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/3.6"

## macOS, OS X, whatever
"https://cran.microsoft.com/snapshot/2020-04-01/bin/macosx/el-capitan/contrib/3.6"

The best way forward, in this case, may be to avoid checkpoint altogether and directly pass the valid URL to available.packages and download.packages:

pkg <- "tidyquant"
contriburl <- "https://cran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/3.6"

db <- available.packages(contriburl)
deps <- tools::package_dependencies(pkg, db, recursive = TRUE)[[pkg]]
download.packages(c(pkg, deps), ".", contriburl = contriburl, type = "win.binary")
Mikael Jagan
  • 9,012
  • 2
  • 17
  • 48
  • This sounds ideal...but when I tried it I get a bunch of warnings about the specified R version not matching the current, but `getOption"repos")` does return `https://mran.microsoft.com/snapshot/2020-04-01`. However when I try to run download.packages() I get `Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1: cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES` which makes sense since there was no version 4.1 then...not sure how to proceed – jerH Feb 08 '22 at 16:37
  • The warning with `r_version = "3.6.3"` is fine if you are just downloading the binaries on your machine, then installing them on machines that actually run R 3.6.3. Unless I'm misunderstanding... It would help to see the code you have tried and your `sessionInfo()`. FWIW, I ran all of the code in my example on a machine running R 4.1.2, with no issues... – Mikael Jagan Feb 08 '22 at 16:46
  • updated the OP with info on what happens when I try.... – jerH Feb 08 '22 at 17:00
  • 1
    Thanks, I'll try to dig into this a bit later today. – Mikael Jagan Feb 08 '22 at 17:04
  • May have cracked it...I just hardcoded the URL into the download.packages() call `download.packages(packages, contriburl="https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/3.6", destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")` – jerH Feb 08 '22 at 17:18
  • Makes sense - in that case you can potentially avoid `checkpoint` altogether. You don't really need the bookkeeping functionality of `checkpoint` at all, now that I think about it. I guess I can update my answer. – Mikael Jagan Feb 08 '22 at 17:23
  • I think you may still need checkpoint so that when you build the list of dependencies you do so for the old version...the 113 tidyquant dependencies from v4.1 is only 87 for 3.6. Many thanks! – jerH Feb 08 '22 at 17:29
  • 1
    I've updated the answer. You don't need `checkpoint`, as long as you pass the correct `contriburl` to _both_ `available.packages` and `download.packages`. BTW, you might consider using the default `which = "strong"`, or equivalently `which = c("Depends", "Imports", "LinkingTo")`, when calling `package_dependencies`. Whether you need `LinkingTo` or not is package-dependent, and if you're not sure, then it might be worth keeping. By my count, it adds 14 dependencies to your 87. – Mikael Jagan Feb 08 '22 at 20:53