1

I am executing a sql query in R using sqldf package to create a data frame in R. But, it is throwing an error:

Error: cannot allocate vector of size 3.9 Gb

I have gone through various threads with a similar issue but I could not find a suitable answer.

Can anyone please help me out on this.

I am using R 2.15.1 version on 64-bit linux machine with 32 GB RAM.

guido
  • 18,864
  • 6
  • 70
  • 95
Kunal Batra
  • 1,001
  • 3
  • 15
  • 23
  • Possible duplicate of [R memory management / cannot allocate vector of size n Mb](http://stackoverflow.com/questions/5171593/r-memory-management-cannot-allocate-vector-of-size-n-mb) – Robert Gowland Apr 15 '16 at 20:45

2 Answers2

3

The error is often misunderstood. It means that R is unable to allocate an additional chunk of 3.9Gb of memory space. If you were to look at the R process, it would have been using a very large amount of the available RAM before it issued the error you saw and you'd have realised that the error meant additional RAM.

You will have to expand upon this in another question to explain what it is you are trying to do as if you can't read data into R with 32Gb of RAM available you will probably need to look at incremental processing of that data. For that we need details of what you are trying to achieve.

Gavin Simpson
  • 170,508
  • 25
  • 396
  • 453
  • I am just trying to run the below query. a<-sqldf("select distinct mp.id as mp_id, sp.id as sp_id, case when mp.brand_id=sp.brand_id then x.weights else 0 end as brand_weight, mp.price_rank as mp_priceRank,sp.price_rank, mp.sp_rank as mp_spRank, sp.sp_rank FROM b as mp join c as sp on sp.id > mp.id and sp.subcategory_id = mp.subcategory_id Left join d as x on mp.subcategory_id=x.subcategory and x.filter_name='Brand'"); When I try to run the same code in SAS which I run on 16 GB RAM windows machine, it works fine. So, I am not able to find out why is it not running on 32 GB RAM machine. – Kunal Batra Sep 17 '12 at 10:20
  • 2
    @KunalBatra R is not SAS and thus SQL via sqldf is not a natural way to transform data; try using indices. Also, you might just made an error in loading your dataset and for instance have numbers stored as strings because of uninterpreted headers or whatever. Anyway, we won't be able to help you without more details. – mbq Sep 17 '12 at 14:55
  • 1
    `sqldf` uses an in memory database by default but you can specify an external one using `sqldf(...whatever..., dbname = tempfile())` . See `?sqldf` and the sqldf home page for more info. – G. Grothendieck Sep 19 '12 at 11:48
0

It's just may be the memory limit in R is too low. First try memory.size() then use memory.limit() to know limit and set new one. I'm not sure if it help. Just let us all know.

Aybek Khodiev
  • 596
  • 1
  • 4
  • 10