0

I'm trying to import a large (14+gb) file into RStudio for use in a project, but I've run into some roadblocks. I installed the 'ff' package to make this easier, but I keep having bugs that I do not know how to fix. Thank you!

This is the code that I used once I imported the ff package:

largeData <- read.csv.ffdf(file="Downloads/measurements.csv")

This is the error message that I get when I run this code:

Error in scan(file = file, what = what, sep = sep, quote = quote, dec = dec, : scan() expected 'a logical', got '"Groningen'

If people know how to do this besides using the ff package, I am open to new ideas.

TylerH
  • 20,799
  • 66
  • 75
  • 101
Andrew Furlong
  • 1
  • 3
    The unmatched quote in `'"Groningen'` suggests to me that your "csv" is somehow malformed, or contains some complex quoting structure for some of the fields. The issue you have right now isn't the size, it's the format of the file. – joran May 21 '19 at 20:22
  • Hi Joran! I slightly altered my code: largeData <- read.table.ffdf(file = "Downloads/measurements.csv", fill = T) I realized that there were many 'n/a' inputs in my file which were throwing an error message. – Andrew Furlong May 21 '19 at 21:13
  • 1
    for faster data import from csv - think about using fread() in data.table package – JMilner May 22 '19 at 00:48
  • This discusses possible solutions for large datasets https://stackoverflow.com/questions/65401851/fast-way-to-download-a-really-big-14-million-row-csv-from-a-zip-file-unzip-an/65402853#65402853 – Andre Wildberg Dec 29 '20 at 14:54

0 Answers0