5

Possible Duplicate:
Trimming a huge (3.5 GB) csv file to read into R

Does R have a good way to transparently deal with data that does not fit into memory? There are a few packages for dealing with big data, but I don't want to make a decision to deploy one without understanding what the actual interface is.

For example, I might have a collection of records that together do not fit into memory. However, if I would load a subset, it is represented by a very simple data frame and I can do all sort of useful selections and aggregations on that data. Is there some sort of package that would allow me to treat the whole collection as a single data frame and perform the same operations on it transparently?

Community
  • 1
  • 1
Serge Levin
  • 51
  • 2
  • 4
    please visit http://cran.r-project.org/web/views/HighPerformanceComputing.html and search within the page for "Large memory" ... it seems as though the `ff` and `bigmemory` packages are what you're looking for. – Ben Bolker Sep 08 '12 at 20:15
  • 1
    @BenBolker Or sqldf. Relevant links: http://stackoverflow.com/questions/4997612/example-of-bigmemory-and-friends-with-file-backing http://stackoverflow.com/questions/3094866/trimming-a-huge-3-5-gb-csv-file-to-read-into-r – Ari B. Friedman Sep 08 '12 at 20:40

0 Answers0