0

I have several large datasets I have been analysing with a script overnight. What I did not consider was the size of the environment I ended up with. It came out to be 22gb and even though i could save it without any problems, it is too big to load again due to lack in ram. Is there a way to save it, maybe loading it in chunks? Or any using another more ram friendly language? Thanks!

r-newbie
  • 11
  • 1
  • 1
    If you have access to a larger machine, perhaps on Azure, Google Cloud or AWS, you could load the file there and save it in parts? Are you sure you're starting with a clean working environment? – Roman Luštrik Jul 02 '22 at 05:59
  • I also learned the painful way to avoid working like that but saving relevant bits of data when they come up. To prevent coming back to work after the weekend and running into issues like yours again. Otherwise, try @RomanLuštrik comment – clemenskuehn Jul 02 '22 at 14:34
  • If those 22GB are for multiple objects, perhaps you can load them one-by-one, or load only those you need. See: https://stackoverflow.com/questions/8700619/get-specific-object-from-rdata-file – Caspar V. Jul 02 '22 at 22:09

0 Answers0