1

I have around 100 csv files with common headers, which i want to merge. The headers are "Lat", "Long" and "value". I am trying to merge all the csv files such that the output would be

"Lat" "Lon" "Value1" "Value2"..."Value 100"

Lat and Lon columns are identical for all of the csv files. Merging two files is easy 

merge(data.frame1, data.frame2, by=c('Lat','Lon'))

However, I tried the following code which didnt work:

file_list <- list.files(~/source)   
list_of_files <- lapply(file_list, read.csv)  
m1 <- merge_all(list_of_files, by=c("Lat","Lon"), all=TRUE)

which throws the error 

Error in merge.data.frame(dfs[[1]], Recall(dfs[-1]), all = TRUE, sort = FALSE,  : 
  formal argument "all" matched by multiple actual arguments.

Can anyone help me in this regard.

csgillespie
  • 59,189
  • 14
  • 150
  • 185
Navin
  • 105
  • 1
  • 4
  • 9
  • 1
    `list.files(~/source)` is a syntax error. `merge_all` doesn't come with the usual R installation; please tell us where it is. – Joshua Ulrich Apr 18 '12 at 12:11
  • merge_all comes with package reshape, and yes sorry with syntax...but in the program i used a correct one. – Navin Apr 18 '12 at 12:18

2 Answers2

4

You can use Reduce and the plain merge:

m1 <- Reduce(function(old, new) { merge(old, new, by=c('Lat','Lon')) }, list_of_files)
huon
  • 94,605
  • 21
  • 231
  • 225
1

This may work as well but you haven't given us any data to work with. I personally use dbaupp's way and am not sure which one is faster; however, I rarely get into big data so the Reduce method is just so much easier for me to work with that's the way I go (I'm releasing a new R package in a few months that has a function to do multimerge that's basically based on the same thinking as dbaupp's response). If you're dealing with big data you may want to bench mark the two (PS I stole this from somewhere as I rarely think in loops to solve problems but can't cite where).

DF <- list_of_files[[1]][, c('lat', 'Lon')]
    for (.df in list_of_files) {
    DF <-merge(DF,.df,by=c('Lat', 'Lon'), all=T, suffixes=c("", ""))
}
DF
Tyler Rinker
  • 108,132
  • 65
  • 322
  • 519
  • Thank you, both works fine, but for merging some csv files i get the error "Error in match.names(clabs, names(xi)) : names do not match previous names". I think this was beacuse of some bug in merge() function reported earlier. – Navin Apr 19 '12 at 08:37