4

I'm currently working on my bachelorthesis and I'm working with RStudio for the first time. I have a small dataset that I'm working with, it's only 20MB. However, when I try to plot it I get the error "can not allocate vector of size 69.2GB" which does not make any sense to me, as my data is not even that big.

I'm on Windows 64-bit and have 16GB RAM which is why I'm also using the 64-bit RStudio version.

I have tried the following things: I typed memory.size() and got 94.6 as a result. I typed memory.limit() and it says: 16314. If I type memory.limit(size = 16000) it says cannot reduce upper memory limit, typing size = 2500 or any other number produces the same error.

Since increasing memory limit in RStudio didn't work I also tried increasing it manually: I rightclicked on RStudio on my Desktop and put --max-mem-size=16000M in the target field, which didn't change anything either.

Here is my code: 

Mois1 <- read.delim("D:/Daten/SoilMoisture/ALL_SM51_SE1_hourly.txt")
Temp1 <- read.delim("D:/Daten/SoilTemperature/ALL_ST51_SE1_hourly.txt")

Mois1 <- rename(Mois1, Date = Date_______Time.UTC. , SWC = SWC.Vol..)
Temp1 <- rename(Temp1, Date = X..Date_______Time.UTC. , Temperature = T..Â.C.)

Mois2019 <- Mois1[1:8756,]
Temp2019 <- Temp1[1:8760,]

plot(Mois2019)

The error occurs after the plot(Mois2019)

I uploaded two sections of the two data files that I'm working with here: https://gofile.io/?c=5crw62 Both documents originally have over 86000 rows each. The two objects Mois2019 and Temp2019 have 8756 and 8760, as you can see in the code.

How can I increase my memory limit?

Phil
  • 85
  • 7
  • Read through the FAQ on this topic. If you don't find something to try, please edit this question (or ask a new one) providing things like (a) a small sample of your data, preferably copy/pasteable like `dput(your_data[1:10, ])` (b) (**important**) the line of code you run that produces the error, and the definitions of any objects other than your data used in that line. – Gregor Thomas Mar 23 '20 at 14:33
  • These problems can generally be divided into 2 classes: (A) you are keeping lots and lots of large data sets around, and you finally reach you memory limit, (B) you do some operation that suddenly makes the memory use explode. Based on the giant 69.2 GB in your error message, it's probably case (B), which is good news because it means the problem *probably* is a single line of code where you do something that tries to create something giant, like an `expand.grid` or a cartesian product join. But we can't do anything but guess without seeing some code and sample data. – Gregor Thomas Mar 23 '20 at 14:39
  • @GregorThomas Post edited. – Phil Mar 23 '20 at 17:15
  • How many columns in your data sets? – Gregor Thomas Mar 23 '20 at 17:28
  • Each data set has two columns. – Phil Mar 23 '20 at 17:29
  • Rather than asking us to go download data, put it in our working directory, and read it in, could you post `dput(Mois2019[1:10, ])`, or if that's too big both `str(Mois2019)` and `dput(droplevels(Mois2019[1:10, ]))`? That will give us a copy/pasteable version of the first 10 rows of your data, preserving all column classes and the structure. It's also nice because it makes the question self-contained, in case the link to your file ever goes down. – Gregor Thomas Mar 23 '20 at 17:33

1 Answers1

3

You are attempting to plot a very large vector (>8500) in three dimensions (Date, Time, SWC) against each other when you are using plot() without any other syntax.

Try to use plot(Mois2019$Date, Mois2019$SWC), and only ask to plot what you want/need.

mhovd
  • 3,724
  • 2
  • 21
  • 47
  • Thank you so much! – Phil Mar 23 '20 at 18:11
  • I'm glad it helped. I also want to plug `ggplot2`, which is package that allows you to create beautiful plots, and is more versatile than the base `plot()`. – mhovd Mar 24 '20 at 10:48