1

I'm running RStudio x64 on Windows 10 with 16GB of RAM. RStudio seems to be running out of memory for allocating large vectors, in this case a 265MB one. I've gone through multiple tests and checks to identify the problem:

  1. Memory limit checks via memory.limit() and memory.size(). Memory limit is ~16GB and size of objects stored in environment is ~5.6GB.
  2. Garbage collection via gc(). This removes some 100s of MBs.
  3. Upped priority of rsession.exe and rstudio.exe via Task Manager to real-time.
  4. Ran chkdsk and RAM diagnostics on system restart. Both returned no errors.

But the problem persists. It seems to me that R can access 16GB of RAM (and shows 16GB committed on Resource Monitor), but somehow is still unable to make a large vector. My main confusion is this: the problem only begins appearing if I run code on multiple datasets consecutively, without restarting RStudio in between. If I do restart RStudio, the problem doesn't show up anymore, not for a few runs.

The error should be replicable with any large R vector allocation (see e.g. the code here). I'm guessing the fault is software, some kind of memory leak, but I'm not sure where or how, or how to automate a fix.

Any thoughts? What have I missed?

Mohammad Ansarin
  • 81
  • 1
  • 13
  • If you show the code, we can possibly help. https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example – Jason Feb 24 '20 at 12:09
  • Also see: https://www.burns-stat.com/pages/Tutor/R_inferno.pdf . Circle 2, "Growing Objects" – Jason Feb 24 '20 at 12:11
  • @Jason, thanks for suggestion. I was able to reproduce the error with the code in the link I mentioned. Just had to repeat the calls more as my memory limit is higher. I don't think there's something in my original code that isn't captured by that example. – Mohammad Ansarin Feb 24 '20 at 12:37
  • What memory usage does `pryr::mem_used()` give? – anotherfred Feb 24 '20 at 13:21
  • @anotherfred, replicating error with the code in the link, replacing matrix row length with 7500000: `pryr`'s `mem_used()` returns 7.69GB while `utils`' `memory.size()` returns 7448MB. Rsession has a commit of ~15GB. Does this answer your question? – Mohammad Ansarin Feb 24 '20 at 15:40

0 Answers0