read multiple CSV and add to list in a loop in R

Question

I have multiple CSV files and want to read them in R. the file names are provided as an argument, so I don't know the names in advance. This is the reason, why I do it in a loop.

Next, I want each dataframe to be appended to a list. As a result, I want to have an indexed list to access access the first line of all dataframes and I think the best way to do this is to have a list of the dataframes available. If there is a more efficient way to do this, any solutions are appreciated.

But for now I will go for the having all dataframes in a list approach. For this I would like to know how I can add a dataframe to a list in a loop.

I have this code:

#splitting the args to get the filenames
splat <- strsplit(args[1], ",")[[1]]
for (fname in splat) {
  #d.fname = dataframe
  d.fname <- read.csv(fname, header = FALSE, sep = ",", dec = ".")
  #code needed to add d.fname to a list?
}

So all in all I have two questions: 1) how can I add dataframes to a list in a loop? 2) is there a better way to do this, with the goal in mind, that later I need to have access to all dataframes like do something (e.g. create a box plot) with the first elements of the first rows of all dataframes?

I am aware of solutions like this: Importing multiple .csv files into R . However, I cannot apply this solution, since I am not able to use list.files(pattern="*.csv") because I dont know the filenames in advance.

you can proceed by `csvNames=strsplit(args[1], ",")[[1]]` then you can simply do `lapply(csvNames, read.csv, header=F, sep=',', dec='.')` — Colonel Beauvel, May 27 '15 at 08:43
next question would be: how to access first elements of first rows of each dataframe in a list? — beta, May 27 '15 at 08:49
thanks. and if i first want to do this for the x[1,1] then for x[2,1] etc... how can this be done without a loop? is there a nice way without a for loop? — beta, May 27 '15 at 08:58
`lapply(seq_along(lst) function(x) lst[[x]][x,1])` ... but try to test and understand lapply first ;) — Colonel Beauvel, May 27 '15 at 09:02

score 3 · Accepted Answer · answered May 27 '15 at 08:54

3

On your original question, you can use directly:

lapply(strsplit(args[1], ",")[[1]], read.csv, header=F, sep=',', dec='.')

answered May 27 '15 at 08:54

Colonel Beauvel

30,423
11
47
87

score 1 · Answer 2 · answered May 27 '15 at 09:46

To answer the first part of your question, you can append to lists in a loop with

# Create an empty list
csvList <- list()
#splitting the args to get the filenames
splat <- strsplit(args[1], ",")[[1]]
for (fname in splat) {
  #d.fname = dataframe
  d.fname <- read.csv(fname, header = FALSE, sep = ",", dec = ".")
  #code needed to add d.fname to a list?
  csvList[[length(csvList)+1]] <- d.fname
}

But I agree lapply is much better and will carry the names through. This can be a useful method for more complicated appending when lapply might start to struggle.

score 1 · Answer 3 · answered Aug 31 '17 at 16:52

That might help - too

# instantiate an empty list
ldf <- list()

# creates the list of all the csv files in the directory 
# filter for filenames holding UnixM
listcsv <- dir(pattern = ".UnixM.") 

# loop thorugh the list holding the single CSV files
for (k in 1:length(listcsv)){
   ldf[[k]] <- read.csv(file = listcsv[k]) 
}

read multiple CSV and add to list in a loop in R

3 Answers3