0

EDIT1: I am using windows Vista and R 2.12.1

I am plotting 84 maps with spplot from the maptools package. I am using two viewports, one to plot the map, and one to plot a trend over time. I have written a loop which goes through each subset of the data, plots the map with countries color coded according to their scores, and plots the trend in that score over iterations. The loop looks like this:

##Read in the shape file, it is 48mb
eu27 <- readShapeSpatial("C:/Users/Thomas/Documents/EU27_shapefile/eu27_cyp.shp",proj4string=CRS     ("+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs"))

##Drop not used variables from the shapefile data table
eu27@data <- data.frame(Order = 1:dim(eu27)[1], Country = as.character(eu27@data[,c("NAME_ISO")]))

##Fix longitude and lattitude for plotting later
xl <- c(-15,36) 
yl <- c(33,73)

##Set the directory for storing the plots
setwd("C:/Users/Thomas/Documents/maps/")

##Define the breaks and  colors
brks <- seq(0,8,1)
colcode <- terrain.colors(length(brks))

##Count used in the loop to keep track of iterations
count <- 1

##Vector to store the trend in time
total <- vector()

##Beginning of loop
for(i in 2001:2003){
 for(j in 1:12){
                ##Subset the data and merge it with the shape file data table
  dat <- d3[d3$Year == i & d3$Month == j, c("No","No.Neg","Country")]
  eu27dat <- merge(eu27@data,dat, by="Country", all.x=TRUE)
  eu27@data <- eu27dat[order(eu27dat$Order),]
  eu27@data$tot <- eu27@data$No + eu27@data$No.Neg

                ##Store the map plot with countries color coded in the plot1 object
  plot1 <- spplot(eu27, "tot", main = paste("Year: ",i,", Month: ",j, sep = ""), xlim=xl, ylim=yl, at = brks, col.regions = colcode)

                ##Remove the variables created
  eu27@data$No <- NULL
  eu27@data$No.Neg <- NULL
  eu27@data$tot <- NULL

                ##Update the vector with trend data from the current iteration
  total[count] <- sum(dat$No.Neg, na.rm = T)

                ##Store the trend data plot in the plot2 object
  plot2 <-xyplot(total ~ 1:length(total), type = "b", xlab = "Time", ylab = "No Votes", ylim = c(-1,4))

                ##Open a PNG device
  png(file = paste("mapNo",count,".png", sep = ""),width=20, height=12, units="cm",bg="white", pointsize=20, res=300)

                ##Create two viewports for the two plots created above
  pushViewport(viewport(layout = grid.layout(1, 2, unit(c(2, 1), "null"))))

                ##Print plot1 in the first viewport
  pushViewport(viewport(layout.pos.col = 1, layout.pos.row = 1)) 
  print(plot1, newpage = F) 
  upViewport() 

                ##Print plot2 in the second viewport
  pushViewport(viewport(layout.pos.col = 2, layout.pos.row = 1)) 
  print(plot2, newpage = F) 
  upViewport()

                ##Close the device
  dev.off()

                ##Update the count
  count <- count + 1
 }
}

The problem is that R crashes after 8 iterations, I suspect that somehow I am using up huge amounts of memory, but I really have no idea of what is going on.

EDIT2: I get a windows error message which says (translated from Germahn): The R for Windows front-end does not function any more

EDIT3: I have been monitoring the memory use from the windows task manager, and after 8 iterations the memory is almost used up completely.

EDIT4: I get the same error when using different graphics devices (png, jpeg, pdf). I was able to run the loop without using the viewports, so I am suspecting that this is related to the viewports.

Best, Thomas

Thomas Jensen
  • 860
  • 3
  • 11
  • 26
  • What do you mean with R crashes? Like it automatically terminates, or returns an error, or even something different? If it is a memory issue and you are using windows, try starting R with `--max-mem-size=2047M` as command line parameter. – Henrik Jan 17 '11 at 12:39
  • By opening the Task manager you can see the amount of memory used (or available) under Performance tab. That should give you some hints if your program indeed has memory issues. – Roman Luštrik Jan 17 '11 at 13:02
  • It might well be the png device. Try using pdf instead. See also : http://stackoverflow.com/questions/4270349/ggsave-png-error-with-larger-size BTW: it's easier if you mention which R version and OS you use. Some things have been fixed. Also the error you get would be interesting. – Joris Meys Jan 17 '11 at 13:06
  • @Roman, thanks for the tip, see edit3 in my question – Thomas Jensen Jan 17 '11 at 14:50
  • @Joris, I get the same error when usimg different graphics devices, could this be related to the use of viewports? (see edit4) – Thomas Jensen Jan 17 '11 at 14:51
  • this is not about plotting "shapefiles", the code plots a Spatial*DataFrame – mdsumner Jan 19 '11 at 22:38

1 Answers1

2

If it works with a small shapefile, but not with a large one (and 48Mb is a large one) then yes, it will be memory. One thing I've seen sometimes help is to stick everything in a loop into a function, so your file looks like this:

for(i in 2001:2003){
 for(j in 1:12){
   doit(i,j,[etc])
  }
}

After each call to doit() lots of things will hopefully be going out of scope and thus garbage collected. I know this trick worked with early versions of R (and Splus) but maybe that's all been fixed now. Check out some of R functions related to memory to get a handle on your process usage. Is this Windows or Unix?

Spacedman
  • 92,590
  • 12
  • 140
  • 224
  • @Spacedman: This is not so much an R issue, but a png issue. Memory management in R ain't going to help much. See also : http://stackoverflow.com/questions/4270349/ggsave-png-error-with-larger-size – Joris Meys Jan 17 '11 at 13:14
  • @Joris That link seems to suggest, to me at least, that the underlying issue is related to memory and is probably to do with being restricted to 2GB (or 4GB on 64 bit Windows) in a 32 bit process. – David Heffernan Jan 17 '11 at 13:53
  • @David : Indeed, it is memory-related, but cannot be resolved in R. Not with setting the limits higher (I had 6Gb at the time I tested), not with using garbage collection, not with cleaning the environment. I worded it wrong maybe, but manual memory management in R is not going to solve it. Updating R hopefully will. – Joris Meys Jan 17 '11 at 14:06
  • @David, @Joris, I am a newbie with regards to garbage collection, i have tried adding the gc() command at the end of each iteration, bit this does not make any difference, are there other ways to make sure the memory is not totally used up? – Thomas Jensen Jan 17 '11 at 14:53
  • @David @Spacedman, I just tried wrapping the content of the loop into a function, and now R does not crash but i get the following error message (translated from German): Can not allocate vector of size 2.5mb – Thomas Jensen Jan 17 '11 at 15:17