How to increase memory limit for R on a Mac?

Question

I've been looking into methods to increase memory limit for R on a Mac, but haven't found anything particularly useful. I tried both Increasing memory limit in R for Mac and R on MacOS Error: vector memory exhausted (limit reached?), but neither worked.

I'm running

fviz_nbclust(df, kmeans, method = "wss")

on a set of data of 1.79m rows and 2 columns. Error says vector memory exhausted (limit reached?). I've tried memory.limit() but this doesn't work on a Mac.

sessionInfo as below:

R version 3.6.1 (2019-07-05)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.6

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] factoextra_1.0.6 ggplot2_3.2.1   

loaded via a namespace (and not attached):
 [1] ggrepel_0.8.1    Rcpp_1.0.3       withr_2.1.2      assertthat_0.2.1
 [5] crayon_1.3.4     dplyr_0.8.3      grid_3.6.1       R6_2.4.1        
 [9] lifecycle_0.1.0  gtable_0.3.0     magrittr_1.5     scales_1.1.0    
[13] pillar_1.4.2     rlang_0.4.1      lazyeval_0.2.2   tools_3.6.1     
[17] glue_1.3.1       purrr_0.3.3      munsell_0.5.0    compiler_3.6.1  
[21] pkgconfig_2.0.3  colorspace_1.4-1 tidyselect_0.2.5 tibble_2.1.3    
>

Can anyone help? Thanks!!

You can refer to the below answer. https://stackoverflow.com/questions/51295402/r-on-macos-error-vector-memory-exhausted-limit-reached Hope it works. — Ashish, Dec 13 '19 at 06:45

mhovd · Answer 1 · 2021-05-04T10:39:25.873

Instead of increasing the memory limit (solutions for which are in your linked questions), you should evaluate your current code. Optimizing code will more often than not make up for limitations in hardware.

There are a few articles (1)(2) on the subject. You should especially look into the packages data.table(3) and/or bigmemory (4) and ff (5). The latter created a virtual data.frame, but can be handled like any regular object in R. Here is an example modified from the latter article:

library(ff)
                                 
# Import the file
bigobj.ff <- read.csv.ffdf(file="bigfile.csv")

# Check the object
class(bigobj.ff)
## [1] "ffdf"

# bigobj.ff is a virtual dataframe, but you can still perform normal operations
sum(bigobj.ff[,3])
## [1] 66029

How to increase memory limit for R on a Mac?

1 Answers1

Linked