3

I am trying to do a dcast in R to generate a matrix as seen in another question I asked

However, I am getting an error:

Error: cannot allocate vector of size 2.8Gb.

My desktop has 8GB of RAM and I am running ubuntu 11.10 64-bit version. Am I perhaps using the wrong version of R? How would I know, is there a way to determine it while running R? I surely must have the necessary space to allocate this vector.

Community
  • 1
  • 1
Dan Q
  • 2,227
  • 3
  • 25
  • 36
  • `sessionInfo()` will tell you if you are using a 32-bit or a 64-bit version of R. – Vincent Zoonekynd Jan 19 '12 at 03:40
  • alright, thanks. Indeed I am running the 64-bit version, so now I wonder why I might not be able to allocate the vector. – Dan Q Jan 19 '12 at 03:47
  • My understanding is that it doesn't matter how much RAM your system has. If you don't have 2.8Gb of _continuous_ address space, you will see this error. – joran Jan 19 '12 at 03:49
  • does that mean there is nothing I can do to alleviate this? Interestingly enough, I tried doing this in R on my macbook air with 4GB of ram and it doesn't give me the error, however it does hang so maybe it would have eventually returned and complained to me about the lack of memory. – Dan Q Jan 19 '12 at 03:58
  • No, there's (almost) always an answer. Memory issues in R are well-trodden ground. For starters, do a search on SO for "[r] cannot allocate" and do some reading. – joran Jan 19 '12 at 04:11
  • I posted an alternative (more memory efficient?) response to your earlier question, but wonder whether you've worked through your whole analysis with one of the smaller data sets? – Martin Morgan Jan 19 '12 at 12:31

1 Answers1

9

The error message means that R needs to allocate another 2.8Gb of memory to complete whatever operation you were trying to perform. It doesn't mean it needed to allocate 2.8Gb maximum. Run top in a shell whilst you run that R code and watch how R uses up memory until it hist a point where the extra 2.8Gb of address space is not available.

Do you have a large swap space on the box. I can easily see how what you are doing uses all 8Gb of RAM plus all your swap space and so there is no other place for R to get memory space from and thus throws the error.

Perhaps you could try doing the dcast in chunks, or try an alternative approach than using dcast. Post another Q if you want help with that.

Gavin Simpson
  • 170,508
  • 25
  • 396
  • 453
  • This could be it, I realize I forgot to allocate a swap partition when I installed ubuntu. Let me try doing that and seeing if that resolves this issue. – Dan Q Jan 19 '12 at 18:40
  • 1
    @DanQ Hmm, unless you have a lot of time to spare or enjoy twiddling your thumbs, this isn't going to a solution - your disc will just thrash as the OS pushes the data R needs back and forwards between the HD and RAM. You should consider if there are more memory efficient ways of doing what you want. – Gavin Simpson Jan 20 '12 at 09:54