0

I'm analyzing the Daily Fishing Effort data from Global Fishing Watch located here. Once extracted, I have a folder with five years worth of daily CSV files named YYYY-MM-DD.csv

How can I automate the import if I only want the files from 2016 and I don't want to have to individually assign each variable a new name? This question seems to imply it, but is more focused on extracting specific columns on import. It would be great to combine an rbind statement and group 2016 into monthly data frames.

mikeLdub
  • 227
  • 3
  • 11
  • Maybe you can get all names of .csv files in your workspace (like this `list.files(pattern="*.csv")`) and then keep only those that match `2016` and import those only. – AntoniosK Oct 15 '18 at 17:33
  • 2
    Suggested duplicate: [How to make a list of data frames](https://stackoverflow.com/a/24376207/903061), I think my answer there should cover all of this. Something like `do.call(rbind, lapply(list.files(pattern = "2016.*csv"), read.csv))` – Gregor Thomas Oct 15 '18 at 17:33
  • 1
    If you need additional help, edit the question with details of what's not working and we can re-open. – Gregor Thomas Oct 15 '18 at 17:37
  • 1
    And I'll amend my above recommendation, with hundreds of CSVs it will certainly be worth it to use `data.table::fread` for speed. Code becomes `library(data.table); result = rbindlist(lapply(list.files(pattern = "2016.*csv"), fread))`. If the individual data frames don't have a `date` column, add `idcol = "date"` as an argument to `rbindlist` and it will automatically create one for you based on the file names. – Gregor Thomas Oct 15 '18 at 17:46
  • Edit: If you need to create the date column, do this: `library(data.table); files = list.files(pattern = "2016.*csv"); data_list = lapply(files, fread); names(data_list) = gsub(".csv", "", files, fixed = TRUE); result = rbindlist(data_list, idcol = "date")`. – Gregor Thomas Oct 15 '18 at 18:08
  • That looks like it @Gregor, apologies for the duplicate, I was using imprecise language in my research. – mikeLdub Oct 15 '18 at 19:29
  • No apologies needed! This question now serves as a helpful pointer to that other one, making it easier to find :) – Gregor Thomas Oct 15 '18 at 19:31

0 Answers0