1

I'm having memory issues when generating many plots and writing them to png/jpeg/eps devices.

require(ggplot2)
...

render <- function(x) {
  fileName=paste(chartDir, "/", x$PACKED[1], ".png", sep="")
  x <- x[,c("EFF_DATE", "variable", "value")]
  png(fileName, width=1920, height=1000, units="px")
  print(qplot(EFF_DATE, value, data = x, facets = variable ~ ., geom="line"))
  dev.off()      
}

d_ply(molten, "PACKED", render, .progress="tk")

The code progresses nicely for the first ~80 plots and then behaves like a fork bomb thereafter, consuming 100% of RAM within a very short time. I've checked the sizes of x supplied to qplot and they're all roughly the same, so it's not the data. The code runs fine when I comment the png line. I get the same issue when I try to use ggsave from the ggplot2 library.

If anyone has an inkling as to why this is happening then I'd love to hear it. However, in anticipation that nobody does, can someone tell me if there is a nice heap analysis tool that I can run inside R to investigate where the memory is going and if there's anything I can do to clean up on the fly? I'd really rather not have to resort to debugging the binary.

Best wishes, Graham.

Graham Bygrave
  • 53
  • 6
  • Argh! - apologies. It does appear to be one specific data.frame that's causing png to blow up. Will investigate further and hopefully complete this thread with something more useful. – Graham Bygrave Feb 01 '11 at 16:54
  • Sounds like you found the problem, but because you are using ggplot2, you could try `ggsave()` instead of `png()`. –  Feb 01 '11 at 17:40

2 Answers2

1

It definitely sounds as though you're running out of memory and swapping to disk.

You can force garbage collection with the gc() function, which takes an optional verbose parameter. Try adding it after dev.off() and see if it helps.

Jeffrey

Jeffrey Breen
  • 463
  • 4
  • 8
  • Searching for '[r] gc' finds a few related answers like http://stackoverflow.com/questions/1467201/forcing-garbage-collection-to-run-in-r-with-the-gc-command – Jeffrey Breen Feb 01 '11 at 16:57
1

Ok it looks as though this is a problem with my call to qplot (bug or pilot error) and not the rendering to png/jpeg/eps as I first thought.

To reproduce, first copy the table listed at the bottom into a file called "bad.data" and then type the following into the R prompt:

  require(ggplot2)
  df_import <- read.table("bad.data", colClasses=c("integer", "POSIXct","factor","integer"), header = TRUE)
  qplot(EFF_DATE, value, data = df_import)



        EFF_DATE variable value
147170 2010-07-05  COUNT_0     0
147254 2010-07-06  COUNT_0     0
147338 2010-07-07  COUNT_0     0
147422 2010-07-08  COUNT_0     0
147506 2010-07-09  COUNT_0     0
147590 2010-07-12  COUNT_0     0
147674 2010-07-13  COUNT_0     0
147758 2010-07-15  COUNT_0     0
147842 2010-07-16  COUNT_0     0
147926 2010-07-19  COUNT_0     0
148010 2010-07-20  COUNT_0     0
148094 2010-07-21  COUNT_0     0
148178 2010-07-22  COUNT_0     0
148262 2010-07-23  COUNT_0     0
148346 2010-07-26  COUNT_0     0
148430 2010-07-28  COUNT_0     0
148514 2010-07-29  COUNT_0     0
148598 2010-07-30  COUNT_0     0
148682 2010-08-02  COUNT_0     0
148766 2010-08-03  COUNT_0     0
148850 2010-08-04  COUNT_0     0
148934 2010-08-05  COUNT_0     0
149018 2010-08-06  COUNT_0     0
149102 2010-08-09  COUNT_0     0
149186 2010-08-10  COUNT_0     0
149271 2010-08-11  COUNT_0     0
149356 2010-08-12  COUNT_0     0
149439 2010-08-13  COUNT_0     0
149521 2010-08-16  COUNT_0     0
149601 2010-08-17  COUNT_0     0
149681 2010-08-18  COUNT_0     0
149761 2010-08-19  COUNT_0     0
149843 2010-08-20  COUNT_0     0
149925 2010-08-23  COUNT_0     0
150004 2010-08-24  COUNT_0     0
150084 2010-08-25  COUNT_0     0
150164 2010-08-26  COUNT_0     0
150245 2010-08-27  COUNT_0     0
150326 2010-08-30  COUNT_0     0
150407 2010-08-31  COUNT_0     0

My architecture is Linux x86_64, although I'm not sure if this is relevant to the problem.

Graham Bygrave
  • 53
  • 6