Unlist multiple csv Files

Question

How can I unlist and rbind the csv files?

I had many csv files in a directory. They all have two columns and unequal number of rows. I made a list using

list.files(pattern=".csv$") # use the pattern argument to define a common pattern  for import files with regex. Here: .csv

list.filenames<-list.files(pattern=".csv$")

list.data<-list()

for (i in 1:length(list.filenames))
{
  list.data[[i]]<-read.csv(list.filenames[i])
}


names(list.data)<-list.filenames

The data look as follows:

$`xxx-xxx.csv`
    PositionX    Value
1           2 -107.068
2           6  -35.074
...
$`xxx1-xxx1.csv`
    PositionX   Value
1           2  -9.230
2           6  -9.230
...

There are thousands of csv files in the format mentioned above. I just would like to rbind them together and, without the header. For example I want to have:

PositionX    Value
1           2 -107.068
2           6  -35.074
3           2  -9.230
4           6  -9.230

I have already tried the following solutions:

big.list.of.data.frames <- lapply(list.data, function(x){
+     read.table(x, skip = 3, header = TRUE, stringsAsFactors = FALSE)
+ })

I get the following:

 Error in read.table(x, header = TRUE, stringsAsFactors = FALSE) : 
  'file' must be a character string or connection

I have also tried

myMergedData <- do.call(rbind,list.data)

I get the error:

Error in rbind(deparse.level, ...) : 
  numbers of columns of arguments do not match

I've made sure all my csv files have 2 columns.

Already tried, I get 'Error in rbind(deparse.level, ...) : numbers of columns of arguments do not match' — Jamil, May 08 '17 at 22:39
Then you don't have the same number of columns for all elements of the list. Check `str(list.data)` to make sure that all elements have only two columns. — Pierre Lapointe, May 08 '17 at 22:42
To get the full `str`: `str(list.data), list.len = length(list.data)))` — Pierre Lapointe, May 08 '17 at 22:48
Position int and value num and all obs have 2 variables. And doing cbind gives `Error in data.frame(..., check.names = FALSE) ` where as r bind gives a different error. — Jamil, May 08 '17 at 22:59

score 1 · Accepted Answer · answered May 08 '17 at 23:41

1

Make a list of read csv files:

list.data <- Map(read.csv, list.files(pattern=".csv$"))

Check if they all have two columns:

all(sapply(list.data, ncol)==2)

If this is not TRUE then investigate which of the files had more or less than 2 columns:

which(sapply(list.data, ncol)!=2)

Otherwise combine as suggested in the comments:

do.call(rbind, list.data)

answered May 08 '17 at 23:41

Karolis Koncevičius

9,417
9
56
89

I get `Error in match.names(clabs, names(xi)) : names do not match previous names` – Jamil May 09 '17 at 00:15
This means that your files have different headers (variable names). Do you get the second line (with `all(`) to return `TRUE` ? – Karolis Koncevičius May 09 '17 at 00:19
Yes I get all(sapply(list.data, ncol)==2) [1] TRUE > > which(sapply(list.data, ncol)!=2) named integer(0) – Jamil May 09 '17 at 00:22
Then your files have different variables names for those two columns. Or so it seems. If you are not afraid to ignore that then you can try this: `list.data <- Map(setNames, list.data, list(names(list.data[[1]])))` before running the last line. – Karolis Koncevičius May 09 '17 at 00:28
I tried I still got the same error – Jamil May 09 '17 at 00:30
It would help if you provided a few examples of how the first few lines of your files look like. Otherwise it's a bit hard to help in the blind. – Karolis Koncevičius May 09 '17 at 00:34

Unlist multiple csv Files

1 Answers1