6

I spotted the following behavior. Say I create the following multi-dimensional array:

spam = array(runif(96*48*60*360), dim = c(96,48,60,360))

It is quite predictable how much memory R should use for this, namely (96*48*60*360) * 4 bytes = 759.4 Mbyte. This is nicely confirmed using the lsos function (see this post):

> lsos()
         Type      Size PrettySize Rows Columns
spam    array 796262520   759.4 Mb   96      48
lsos function       776  776 bytes   NA      NA

R as a process however uses much more memory, roughly twice the size:

$ top | grep rsession
82:17628 hiemstra  20   0 1614m **1.5g** 8996 S  0.3 40.4   0:04.85 rsession

Why does R do this? I assume the extra reserved memory is allocated to make it more quickly accessible to R? Any thought's?

Community
  • 1
  • 1
Paul Hiemstra
  • 59,984
  • 12
  • 142
  • 149

1 Answers1

6

Because the garbage collector has not run yet.
So there's a lot of garbage, probably generated during the creation of the big array, that has to be cleared.

If you force a garbage collection by calling gc() function, you will see that the used memory will be pretty near to the size of your array:

> memory.size()
[1] 775.96
digEmAll
  • 56,430
  • 9
  • 115
  • 140
  • So in the end the array uses 759.4 Mbytes, but during the creation it uses more? That could be unfortunate if the array fits in memory, but the spike in memory usage during creation uses more than the available amount of memory. – Paul Hiemstra Jul 26 '12 at 08:44
  • 1
    Well, I don't know exactly what happens under the hood, but with your code you're not simply allocating an array; in fact, first you generate a vector of random numbers, then you allocate an array by copying that values. So, I guess most of the overhead (namely garbage) is due to that throwaway vector... – digEmAll Jul 26 '12 at 08:52
  • However, I guess the garbage collector triggers automatically when you're out-of-memory so I don't think that will be performance issues as far as the array fits in memory... – digEmAll Jul 26 '12 at 08:55