for loop to read.csv() several CSV at the same time

Question

I'm looking to loop through 4 different folders of CSV files, assign them to different objects, and use all 4 data frames in a later function. All folders have the same number of CSV files (400).

milk_contents <- list.dir(milk folder)
cereal_contents <- list.dir (cereal)
OJ_contents <- list.dir(OJ)
cat_contents <- list.dir(cat)
milk_loc <- "C/Desktop/milk"
cereal_loc <- "C/Desktop/cereal"
OJ_loc <- "C/Desktop/OJ"
cat_loc <- "C/Desktop/cat"

something like:

for(sheet in milk){
milk_sheet <- read.csv(paste0(milk_loc,"/",sheet),stringasfactors = false)
   for(file in cereal){
       cereal_file <- read.csv(paste0(cereal_loc,"/",file),stringasfactors = false)
      for(cup in OJ){
       OJ_cup <- read.csv(paste0(OJ_loc,"/",cup),stringasfactors = false)
           for(paw in cat){
       cat_paw <- read.csv(paste0(cat_loc,"/",paw),stringasfactors = false)
       ex_function(milk_sheet, cereal_file, OJ_cup, cat_paw)
}}}}

I'm thinking there's probably a more elegant solution out there -

Possible duplicate of [Importing multiple .csv files into R](https://stackoverflow.com/questions/11433432/importing-multiple-csv-files-into-r) — csgroen, Feb 02 '18 at 15:50
@csgroen - related, however, my question asks for csv from *different* folders - please read the question before you comment. — longlivebrew, Feb 02 '18 at 15:58
In fact, if you do it like that, you would end up calling ex_function 400^4 times. Is that what you want to do? or should ex_function be called 400 times instead? — loki, Feb 02 '18 at 16:02
It uses all the data frames and re-formats the data into a later function. I don't believe it should matter, though. All I need is the separate csvs read into R. — longlivebrew, Feb 02 '18 at 16:03
It should be called 400 times. The for loop is just to open the CSV files. I cannot load them in at the same time, that's 100 GB of data and my computer only has 16 gb ram. — longlivebrew, Feb 02 '18 at 16:03
are the csv files all named the same? (Do all folders each hold "file1.csv", "file2.csv" and so on?) — loki, Feb 02 '18 at 16:04
The for loop should just read in each of the 4 CSV (from each folder), and then the function will operate 400 times. — longlivebrew, Feb 02 '18 at 16:04
No, they are different data sets that needed to be combined in the later function, so they are named differently - they also are different lengths :P — longlivebrew, Feb 02 '18 at 16:05
But they must follow some pattern, otherwise they would be combined randomly, in the worst case. Do I get that right? It would be easier if you could share some insights into the names of these files. — loki, Feb 02 '18 at 16:07
Sort of...they are named (1000 Status, 1000 Temp, 1000 Status2, 1000 OA Temp, 2000 Status, 2000 Temp, 1000 Status 2, 1000 OA Temp, etc.) - they should come in in order, I would guess — longlivebrew, Feb 02 '18 at 16:11

score 1 · Accepted Answer · answered Feb 02 '18 at 19:21

Right now you have set up an combinatorial set of loops, i.e. each every file will be analyzed with every other file (from their respective folders) by ex_function. But it sounds like you want to run ex_function 400 times, i.e. on each set of four files grouped by their position i the directory. If this is an accurate assessment of the situation read on, if not please clarify.

The simplest way to do this is to loop through the index of one directory and open all four files at the same index position (this assumes file 1 in milk needs to be compared with file 1 of cat, oj and cereal).

for (i in seq_along(milk_contents)){
  milk_sheet <- read.csv(paste0(milk_loc,"/",milk_contents[i]),stringasfactors = false)
  cereal_file <- read.csv(paste0(cereal_loc,"/",cereal_contents[i]),stringasfactors = false)
  OJ_cup <- read.csv(paste0(OJ_loc,"/",OJ_contents[i]),stringasfactors = false)
  cat_paw <- read.csv(paste0(cat_loc,"/",cat_contents[i]),stringasfactors = false)
  ex_function(milk_sheet, cereal_file, OJ_cup, cat_paw)
}

This assumes that each directory has the exact same number of files.

I was just thinking something like this!! thank you for articulating it better! :) — longlivebrew, Feb 02 '18 at 19:51

for loop to read.csv() several CSV at the same time

1 Answers1