3

I am wanting to clip a large shapefile (67MB) in program R and derive a much smaller raster from around ~5% of it. Once loaded the shapefile has 221388 features and 5 fields - and explodes to 746 MB.

My difficulty comes when trying to clip the file to a workable size - the program crashes after a few minutes. I have tried both crop (from raster) and gIntersection (from rgeos) without success. I have 8GB of RAM - clearly there is a memory issue.

I am guessing there maybe a work around. I know that there are some big memory packages out there - but can any of them help in my kind of situation? My current code is below:

# dataset can be found at 
# http://data.fao.org/map?entryId=271096b2-8a12-4050-9ff2-27c0fec16c8f

# location of files
ogrListLayers("C:/Users/Me/Documents/PNG Glob")

# import shapefile
ogrDrivers()[10,]

# shapefiles
Glob<-readOGR("C:/Users/Me/Documents/PNG Glob", layer="png_gc_adg_1")

# assign projection
Glob@proj4string<- CRS("+proj=longlat")

#object size
object.size(Glob) 

# clipping
crop(Glob, extent(c(144,146,-7,-5)))
Nathan
  • 646
  • 5
  • 10
  • 2
    The answer to [this question](http://stackoverflow.com/questions/18319106/gdal-ogr-how-to-really-crop-a-shapefile) might help you. But I am not sure about the feasibility on Windows. –  Feb 13 '15 at 06:15
  • Thanks Pascal - I'm investigating this now. – Nathan Feb 17 '15 at 06:08

1 Answers1

1

As suggested by @Pascal, GDAL's ogr2ogr is useful for this. You can call it from R with system as follows (including on Windows), though this assumes that (1) you have a working GDAL installation and (2) the path to the GDAL binaries exists in your PATH environment variable:

  1. Download and unzip the PNG shapefile:

    download.file('http://www.fao.org/geonetwork/srv/en/resources.get?id=37162&fname=png_gc_adg.zip&access=private', 
              f <- tempfile(fileext='.zip'))
unzip(f, exdir=tempdir())

  2. Call ogr2ogr with system from R to clip the PNG shapefile and save the resulting .shp to the working directory:

    system(sprintf('ogr2ogr -clipsrc 144 -7 146 -5 png_clip.shp %s', 
               file.path(tempdir(), 'png_gc_adg_1.shp')))

    On my system this took around 70 seconds, and memory usage didn't seem to increase by more than about 100MB. (I did get a lot of warnings of the likes of Warning 1: Value 138717513240 of field AREA of feature 0 not successfully written. Possibly due to too larger number with respect to field width - not sure what that's about.)

  3. Load the clipped shapefile: 

    library(rgdal)
p <- readOGR('.', 'png_clip')
plot(p)

    png clip

jbaums
  • 27,115
  • 5
  • 79
  • 119
  • Thanks for this! This looks the way to go. I'm currently trying to install GDAL but having a few installation issues... once it is up and running I'll give it a go. – Nathan Feb 17 '15 at 06:08
  • @NathanWhitmore, I was afraid of these issues on Windows OS. –  Feb 17 '15 at 06:25
  • @Pascal - It can be a pain, but it _is_ possible. I am using Windows :) – jbaums Feb 17 '15 at 06:27
  • @jbaums Nice to hear. Thus there is a hope for the OP :) –  Feb 17 '15 at 06:28
  • 1
    @jbaums Finally got it running - thanks. After more than a little confusion I managed to get ogr2ogr up running using OSGeo4W. The R code for getting the path right is here: http://gis.stackexchange.com/questions/79981/how-can-the-ogr2ogr-help-to-merge-a-shp-file/80005#80005 – Nathan Feb 17 '15 at 08:09