how to average values across data frames?

Question

file.names <- list.files(path = 'mypath')
file.names <- paste("mypath", file.names, sep="/")
for(i in 1:length(file.names))
{
   assign(paste("Frame",i,""), read.table(file.names[i], sep="", header=FALSE))
}

My above code reads files from a directory and adds them to a data frame. I have thousands of these files. The question is how can i get all the data frames that i create for each file and average each value across all data frames. Its just like when you have a 100x 100 matrix of 1000 files (dataframes) you just want one 100 x 100 matrix with average values across the dataframes. Any help is really appreciated. I have been stuck for a while with this.

Why put them in separate data frames? At least [put them all in a single list of data frames](https://stackoverflow.com/a/24376207/903061), or if they have the same columns put them in a single data frame with a `file` column indicating the source. — Gregor Thomas, Sep 19 '18 at 02:24
Might be useful https://stackoverflow.com/q/19218475/ & https://stackoverflow.com/q/44211911/ — Tung, Sep 19 '18 at 03:41
all my files have the same rows and columns and i do know how to add them to one data frame but then i dont know how to differentiate between files and then average suppose [1,1] across all files!! Thanks guys — RnD, Sep 19 '18 at 04:11
@Gregor i followed your suggestion and putting all the dataframes in a list of dataframes but now how can i average the values across the dataframes. Please help...! — RnD, Sep 19 '18 at 18:05
If you want the averages in each cell, then `Reduce("+", list_of_data) / length(list_of_data)`. If you have trouble with that, please edit your question to make a small reproducible example. — Gregor Thomas, Sep 19 '18 at 18:13

score 0 · Accepted Answer · answered Sep 20 '18 at 01:35

The following code seem to do the trick. Thanks to @Gregor

X <- NULL
mylist <- list()
args = commandArgs(trailingOnly=TRUE)
# test if there is at least one argument: if not, return an error
if (length(args)==0){
  stop("At least one argument must be supplied (input file).n", call.=FALSE)
} else if (length(args)==1){
  file.names <- list.files(path =args[1],pattern=".gdat")
  file.names <- paste(args[1], file.names, sep="/")
  args[2] <- paste(args[1], "avg.txt", sep="/")

  for(i in 1:length(file.names))
  {mylist[i]  <- list(read.table(file.names[i], sep="", header=FALSE))}
  X <- Reduce("+", mylist) / length(mylist) #this is the funx that averages across dataframes
  write.table(X, file=args[2], sep="\t",row.names=FALSE, quote=FALSE)
}

how to average values across data frames?

1 Answers1