4

I would like to read a gzipped GeoTIFF from a server without downloading it. I just don't want to create a lot of temporary files that I have to delete later on.

I see it is possible with .csv.gz files.

With download I do it in the following way:

library(raster)
link <- "ftp://ftp.glcf.umd.edu/glcf/SRTM/Degree_Tiles/n000/SRTM_ff03_n000e010
/SRTM_ff03_n000e010.tif.gz"

download.file(link, "test.tif.gz")
gunzip("test.tif.gz")
myras <- raster("test.tif")
plot(myras)

I can read an uncompressed file directly from a link:

link <- "http://download.osgeo.org/geotiff/samples/usgs/o41078a5.tif"
myras <- raster(link)
plot(myras)
myextent <- drawExtent()
plot(myras, ext=myextent)

Here I realize that it might not be a good Idea to not download it to local storage, because I assume that every action you subsequently do with myras needs the data to flow over the internet again. But anyway, just for proof of concept I would like to do it. And there are cases where you just want to display the TIFF without doing any furher calculations with it and therefore don't want to create a temporary file for it.

To read the (downloaded) tiff.gz file without uncompressing it first I tried:

> raster(gzfile("test.tif.gz"))
Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘raster’ for signature ‘"gzfile"’

To read a tiff.gz file directly from the server with a connection I tried the following:

> con <- gzcon(url("ftp://ftp.glcf.umd.edu/glcf/SRTM/Degree_Tiles/n000/SRTM_ff03_n000e010/SRTM_ff03_n000e010.tif.gz"))
> raster(con)
Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘raster’ for signature ‘"gzcon"’

> raw <- textConnection(readLines(con))
> raster(raw)
Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘raster’ for signature ‘"textConnection"’

> rawBin <- textConnection(readBin(con))
Error in readBin(con) : argument "what" is missing, with no default

> con <- gzfile("ftp://ftp.glcf.umd.edu/glcf/SRTM/Degree_Tiles/n000/SRTM_ff03_n000e010/SRTM_ff03_n000e010.tif.gz")
> myras <- raster(con)
Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘raster’ for signature ‘"gzfile"’

I found this Stackoverflow question about how to read a zipped binary file connection, but I am not sure whether and GeoTIFF is binary (is it?) and which parameters to pass to the readBin() function.

I feel like randomly trying things out because I don't really understand how connections work. Can anyone help me with this?

Community
  • 1
  • 1
nnn
  • 4,985
  • 4
  • 24
  • 34

2 Answers2

0

I found a solution for this, maybe you have too.

The function readTIFF() from the tiff package can read a tiff image from a raw vector. So you can read your connection into a raw vector with readBin(), then read that raw vector with readTiff().

# create connection to a gz file
con <- gzfile("test.tif.gz", open = "rb")

# read data from this connection into a raw vector
myras.raw <- readBin(con, what = "raw", n = 1e10)

# read this raw vector
myras <- readTIFF(myras.raw)

I hope this can help :)

norival
  • 49
  • 6
0

Another solution is that the underlying GDAL library provides some specific 'virtual' file systems that allow a zipped and/or remote resource to be read.

https://gdal.org/user/virtual_file_systems.html

The raster command supports those, basically passing the provided file path directly to GDAL to read the data. So you should be able to use the following to load the data directly.

library(raster)
link <- "/vsigzip//vsicurl/ftp://ftp.glcf.umd.edu/glcf/SRTM/Degree_Tiles/n000/SRTM_ff03_n000e010
/SRTM_ff03_n000e010.tif.gz"
David_O
  • 1,143
  • 7
  • 16
  • Seems to be something missing in the comment about using gdal file system. Once you assign the path to link do you mean that raster(link) should be able to read it? I'd like to use this on a local gz file but haven't had any success. – JerryN May 02 '20 at 18:08
  • Was a bit of a sparse answer, true! Yes, you can use the link with `raster(link)` but for a local file, you don't need the `vsicurl` bit, which helps with remote data. So: `link <- "/vsigzip/DigitalTerrain_BNG.tif.gz"; r <- raster(link)` If there is more than one image in the archive, it gets more interesting, but that works with a single gzipped raster file. – David_O May 04 '20 at 14:50
  • At the goal link above a couple of options are presented. /vsigzip/my.gz # (relative path to the .gz) /vsigzip//home/even/my.gz # (absolute path to the .gz) – JerryN May 04 '20 at 15:41
  • Here's a relative version. `link = "/vsigzip/Barley.crop.calendar.fill.nc.gz"; raster(link)` returns `Error in .rasterObjectFromFile(x, band = band, objecttype = "RasterLayer", : Cannot create a RasterLayer object from this file. (file does not exist)`. However, when I click the .gz file the .nc file is automatically extracted. When I attempt to load the .nc file with raster, it returns an error that I need to choose a variable but does recognize it. – JerryN May 04 '20 at 16:06
  • I wonder if my issue is because the file inside the gz file is a netcdf file rather than a tif file – JerryN May 04 '20 at 20:38