How can i solve memory limit issue in r

Question

I am worknig with 2 dataframes in r and when I try to merge them there is an erro: Error: cannot allocate vector of size 565.3 Mb

If I check: memory.limit()

memory.limit() [1] 1.759219e+13

My PC 8GB RAM.

What can I do to solve this issue??

Are you sure you correctly specified the columns you want to join on? If you have duplicates in the column you want to join on the data can grow very large very quickly. — MrFlick, May 28 '19 at 14:15

score 1 · Answer 1 · answered May 28 '19 at 11:39

1

If you need to merge large data frames in R, one good option is to do it in pieces of, say 10000 rows. If you're merging data frames x and y, loop over 10000-row pieces of x, merge (or rather use plyr::join) with y and immediately append these results to a sigle csv-file. After all pieces have been merged and written to file, read that csv-file. This is very memory-efficient with proper use of logical index vectors and well placed rm and gc calls. It's not fast though.

answered May 28 '19 at 11:39

NK09

195
1
12

I didn´t think about this solution and it works for me. thnks – s157 May 29 '19 at 15:04
can u accept the solution by marking the flag – NK09 May 30 '19 at 06:46

How can i solve memory limit issue in r

1 Answers1