R: save each loop result into one data frame

Question

I have written a loop in R (still learning). My purpose is to pick the max AvgConc and max Roll_TotDep from each looping file, and then have two data frames that each contains all the max numbers picked from individual files. The code I wrote only save the last iteration results (for only one single file)... Can someone point me a right direction to revise my code, so I can append the result of each new iteration with previous ones? Thanks!

data.folder <- "D:\\20150804"
files <- list.files(path=data.folder)

for (i in 1:length(files)) {
  sub <- read.table(file.path(data.folder, files[i]), header=T)
  max1Conc <- sub[which.max(sub$AvgConc),]
  maxETD <- sub[which.max(sub$Roll_TotDep),]
  write.csv(max1Conc, file= "max1Conc.csv", append=TRUE)
  write.csv(maxETD, file= "maxETD.csv", append=TRUE)
 }

Have a look at [this](http://stackoverflow.com/questions/30835815/save-imported-csv-data-in-vector-r/30835924#30835924) old solution of mine. It can be adapted easily for data frame I guess. Just a combination of `paste` and `assign`. — SabDeM, Aug 06 '15 at 20:07

score 0 · Answer 1 · answered Aug 06 '15 at 16:04

The problem is that max1Conc and maxETD are not lists data.frames or vectors (or other types of object capable of storing more than one value).

To fix this:

maxETD<-vector()
max1Conc<-vector()
for (i in 1:length(files)) {
  sub <- read.table(file.path(data.folder, files[i]), header=T)
  max1Conc <- append(max1Conc,sub[which.max(sub$AvgConc),])
  maxETD <- append(maxETD,sub[which.max(sub$Roll_TotDep),])
  write.csv(max1Conc, file= "max1Conc.csv", append=TRUE)
  write.csv(maxETD, file= "maxETD.csv", append=TRUE)
 }

The difference here is that I made the two variables you wish to write out empty vectors (max1Conc and maxETD), and then used the append command to add each successive value to the vectors.

There are more idiomatic R ways of accomplishing your goal; personally, I suggest you look into learning the apply family of functions. (http://adv-r.had.co.nz/Functionals.html)

ulfelder · Answer 2 · 2015-08-06T17:17:36.863

I can't directly test the whole thing because I don't have a directory with files like yours, but I tested the parts, and I think this should work as an apply-driven alternative. It starts with a pair of functions, one to ingest a file from your directory and other to make a row out of the two max values from each of those files:

library(dplyr)
data.folder <- "D:\\20150804"

getfile <- function(filename) {
    sub <- read.table(file.path(data.folder, filename), header=TRUE)
    return(sub)
}

getmaxes <- function(df) {
    rowi <- data.frame(AvConc.max = max(df[,"AvConc"]), ETD.max = max(df[,"ETD"]))
    return(rowi)
}

Then it uses a couple of rounds of lapply --- embedded in piping courtesy ofdplyr --- to a) build a list with each data set as an item, b) build a second list of one-row data frames with the maxes from each item in the first list, c) rbind those rows into one big data frame, d) and then cbind the filenames to that data frame for reference.

dfmax <- lapply(as.list(list.files(path = data.folder)), getfiles) %>%
    lapply(., getmaxes) %>%
    Reduce(function(...) rbind(...), .) %>%
    data.frame(file = list.files(path = data.folder), .)

R: save each loop result into one data frame

2 Answers2