0

I have a CSV file. Its size is 300MB. I want to do FFT on each column using this command:

 apply(df,2,function(x){fft(x[!is.na(x)])})

but I get this error:

Error: cannot allocate vector of size 293.0 Mb
In addition: Warning messages:
1: In as.matrix.data.frame(X) :
  Reached total allocation of 1535Mb: see help(memory.size)
2: In unlist(X, recursive = FALSE, use.names = FALSE) :
  Reached total allocation of 1535Mb: see help(memory.size)
3: In unlist(X, recursive = FALSE, use.names = FALSE) :
  Reached total allocation of 1535Mb: see help(memory.size)
4: In aperm.default(X, c(s.call, s.ans)) :
  Reached total allocation of 1535Mb: see help(memory.size)
5: In aperm.default(X, c(s.call, s.ans)) :
  Reached total allocation of 1535Mb: see help(memory.size)
6: In aperm.default(X, c(s.call, s.ans)) :
  Reached total allocation of 1535Mb: see help(memory.size)
7: In aperm.default(X, c(s.call, s.ans)) :
  Reached total allocation of 1535Mb: see help(memory.size)

I am using Windows 7-32 bit and if I run this command memory.limit() the result is 3000

How can I solve my problem? I can't buy more RAM ;)

Ferdinand.kraft
  • 12,579
  • 10
  • 47
  • 69
TangoStar
  • 551
  • 2
  • 8
  • 24

1 Answers1

0

Your memory doesn't need to only fit the newly allocated vector, but also everything else that's in your R session. That's why you get that error even though your limit looks OK.

I suggest using a for() loop instead of apply(). This is maybe less elegant, but more memory-efficient, since you only have one column at a time in memory

FrankK
  • 1