6

When R boots, the memory limit (as returned by memory.limit) is set to 2GB, regardless of the available memory on the computer. (I found that out recently). I imagine that at some point in the booting process, this limit is set to the actually available memory.

This can be seen by printing memory.limit() in the .Rprofile file which is sourced at startup. It prints "2047". On the other hand, when R has booted and I type memory.limit() in the console, I get "16289".

I use a custom .Rprofile file and I need to have access to more than 2GB during bootup.

How can override this limit?

My current workaround is to set the limit myself in the .Rprofile using memory.limit(size=16289) but then I will have to edit this every time I work on a computer with a different amount of RAM which happens fairly often.

Is there an option I can change, a .ini file I can edit, or anything I can do about it?

OS and R version:

> sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

Edit: this is not a duplicate, at least not a duplicate of the proposed question. It is not about managing available memory! I have 16GB of memory and memory.limit() shows that my limit is indeed 16GB.

It all started when I got the warning that I had "reached 2GB memory allocation" (implicating that I had a 2GB memory limit). After investigation, it appears that indeed R limits the memory at 2GB during the startup process.

I want to load my data automatically when R starts, for this I have a small loading script in the .Rprofile. I load more than 2GB data hence I need to have access to my 16GB. My question is about achieving this. This has nothing at all in common with the proposed duplicate, except keywords...

Community
  • 1
  • 1
asachet
  • 6,620
  • 2
  • 30
  • 74
  • 1
    Which R version and OS are using? – Carlos Cinelli Dec 14 '15 at 12:52
  • 3
    @Vova Not a duplicate, the issue is completely different IMO. I do have enough memory and I am not doing any memory hungry operation. My problem is that the memory limit is set to 2GB when R boots. – asachet Dec 14 '15 at 13:33
  • 1
    I must say that I am surprised at the number of "close" vote... The so-called duplicate question is about someone reaching the memory limit on their PC. The "solution" is to optimize the code to be less memoryvore. ---- My problem is that during R startup, the memory limit is set artificially low, therefore preventing me from loading more than 2GB data automatically from the .Rprofile file. It is absurd to refer me to the other post: I CAN load the data once R has started because i have waaaay enough memory. My question is about the memory limit of 2GB at startup, this is completely different. – asachet Dec 14 '15 at 17:19
  • 1
    Wrt to the "duplicate" question: my memory is not "fragmented" because R has just started and the global environment is empty. R does not "compete" with other programs because I have >10GB RAM free. My question being about R memory limit does not make it a dupe. – asachet Dec 14 '15 at 17:22

1 Answers1

1

I'm interpreting this as you wanting memory.limit(size=16289) in your .RProfile file, but you don't want to set the specific number every time you change computers with different memory. Why not just dynamically pull the memory you need? In windows:

TOT_MEM <- as.numeric(gsub("\r","",gsub("TotalVisibleMemorySize=","",system('wmic OS get TotalVisibleMemorySize /Value',intern=TRUE)[3])))/1024
memory.limit(size=TOT_MEM)

which would set the available memory to the total memory of the system, or 

FREE_MEM <- as.numeric(gsub("\r","",gsub("FreePhysicalMemory=","",system('wmic OS get FreePhysicalMemory /Value',intern=TRUE)[3])))/1024
memory.limit(size=FREE_MEM)

which would set memory.limit to the total available memory on boot.

Place this in RProfile, above where you load your data.

Chris
  • 6,302
  • 1
  • 27
  • 54
  • Thanks for the system call! It will require some adaptation though, as I use Windows and Linux interchangeably... I guess I can test for the OS and branch accordingly. I keep thinking that there must be some kind of .ini file where I could specify what the memory limit at startup should be. – asachet Dec 15 '15 at 10:19
  • (Now that I think about it, I haven't even checked if the issue appears on Linux too) – asachet Dec 15 '15 at 10:20