1

In a shiny app I would like users to be able to download a remote (.nc) file with file size limit of say, 100MB.

For some files inspecting the header works well, as suggested here https://stackoverflow.com/a/20921907/6424231 e.g.

httr::headers(httr::HEAD("https://www.unidata.ucar.edu/software/netcdf/examples/ECMWF_ERA-40_subset.nc"))[["Content-Length"]]
#> [1] "22165040"

But for .nc files on THREDDS servers the Content-Length property isn't available e.g.

httr::headers(httr::HEAD("https://dapds00.nci.org.au/thredds/ncss/uc0/Test_pixel_count.nc?var=Band1&north=-22.9556&west=142&east=143&south=-25.0706&disableProjSubset=on&horizStride=1"))[["Content-Length"]]
#> NULL

So I have no idea of the size of this file before downloading.

Is there a way I could allow download.file() to start but then abort the download if it reaches 100MB and the download hasn't finished? I know there are also timeout options available in httr but ideally I would like the limit to be based on file size since I don't want users with slow connections to be prevented from downloading relatively small files.

Is this possible in R or are there any other ways to achieve this to avoid users downloading arbitrarily large files?

Danny
  • 448
  • 3
  • 15
  • 1
    I'd suggest you download the files with `httr::GET(..., httr::write_disk())` and check if there are ways to configure download limits there. `curl::curl_options("maxfilesize")` looks promising. It should be compatible with `httr::config()` – Gregor de Cillia Jul 23 '21 at 17:13
  • Thanks, I'll look into this. Would `httr::GET` also work on Windows without users needing to install additional things (apart from other R packages)? I know it depends on Curl somehow. – Danny Jul 25 '21 at 13:33

0 Answers0