3

How can I create a large matrix in R or how can I zip a large matrix into some object?

y <- matrix(rnorm(2e4*2e4),1:20000,1:20000)
Error: cannot allocate vector of size "..." Gb

How is it possible to allocate this matrix?

tshepang
  • 12,111
  • 21
  • 91
  • 136
And_R
  • 1,647
  • 3
  • 18
  • 32
  • 1
    I've voted to close this since it isn't clear what your problem is. Do need want help with how to install the `bigmemory` package, or a recommendation on how to deal with big datasets? – Richie Cotton Feb 07 '14 at 13:31
  • For what it's worth, using 64-bit R, installing more RAM, and reading http://cran.r-project.org/web/views/HighPerformanceComputing.html will get you a long way. – Richie Cotton Feb 07 '14 at 13:32

2 Answers2

3

I'm not sure why I'm giving this a serious answer but here goes...

1.I can install bigmemory just fine. Perhaps if you are having problems you should share the error message, your environment and OS, and then ask for help on that instead.

2.Perhaps this is just an example:

  y <- outer(1:20000,1:20000)
  Error: cannot allocate vector of size "..." Gb

but do you really want to do this? Couldn't you refactor your code to iterate/resample a smaller matrix many times or read the data a bit at a time, to wit...

3.Have you considered using RSQlite? It's super easy to install and especially now with the dplyr package is very user friendly.

Stephen Henderson
  • 6,340
  • 3
  • 27
  • 33
  • 1
    y <- outer(1:20000,1:20000) was just an example. I'm working with large matrixes. And the problem I have by install.packages("bigmemory") is that R cannot find the particular package in CRAN. You can check, if you don't believe me. – And_R Feb 07 '14 at 13:39
  • 1
    @And `bigmemory` is on CRAN, but doesn't run under Windows. See http://cran.r-project.org/web/packages/bigmemory/index.html – Richie Cotton Feb 07 '14 at 14:24
  • As a good alternative of "bigmemory" is "ff" package. – And_R Feb 07 '14 at 14:48
2

I am a huge fan of the bigmemory package and I am a Windows user. To use the package I first chose to stay with R 2.15 as it still has a functional Windows version of bigmemory package available on CRAN. For R 3.0 I had to compile that old version to work with my big matrices.

Waldir Leoncio
  • 10,853
  • 19
  • 77
  • 107
Andrey Shabalin
  • 4,389
  • 1
  • 19
  • 18