0

Hi everyone a have a dataframe of dimension 1x147,216 (quite large) and I want to calculate a correlation matrix:

similarity = cor(datExpr, method="spearman")

but I receive the error message

"Error: cannot allocate vector of size 161.5 Gb"

I checked all the following recommended solutions but none worked:

memory.limit(size=)

or

if(.Platform$OS.type == "windows") withAutoprint({

  memory.size()
  memory.size(TRUE)
  memory.limit()

})

memory.limit(size=170000000)

Anyone has any idea? Has anyone used a sparse matrix for solving this?

Leonardo
  • 2,439
  • 33
  • 17
  • 31
KLB
  • 1
  • Does calculating correlation between two vectors with one value each make sense to you? – CJR Oct 23 '21 at 22:37
  • Yes, you are right , I meant to write a dataframe of dimension 10 x147,216 – KLB Oct 24 '21 at 23:09
  • Well if you want a 147,216 x 147,216 correlation matrix you're gonna need to store all those numbers. You should probably think about what you're doing next (if you just want to find the highest correlations you might have some options by only storing what you need). – CJR Oct 24 '21 at 23:48

0 Answers0