4

I am trying to read and merge together all the csv files in a directory. I found this excellent SO answer: Importing multiple .csv files into R but it doesn't seem to work for me.

I am able to list the files (they are in my home directory in a sub-folder called "test"):

library(data.table)  
files <- list.files(path = "test",pattern = ".csv")
print(files)

This correctly prints the content of the directory.

When I try to load them using

temp <- lapply(files, fread, sep=",")
data <- rbindlist(temp)

I get File 'xyz.csv' does not exist. Include one or more spaces to consider the input a system command.

Do I have to somehow specify the path again? I fought that this information is already contained in the file object. Thanks for any help!

Community
  • 1
  • 1
Smajl
  • 7,555
  • 29
  • 108
  • 179
  • I guess the error is in the `temp <- lapply(files ,...)` step? Can you try with `library(readr); lapply(files, read_csv)` – akrun Jan 20 '16 at 09:53
  • @akrun Yes, the error is on that particular line – Smajl Jan 20 '16 at 09:53
  • I don't know if this is some delimiter issue, can you try `library(readr); lapply(files, read_csv)` or even the `base R`, i.e. `lapply(files, read.csv, header=TRUE, stringsAsFactors=FALSE)` – akrun Jan 20 '16 at 09:54
  • I can't seem to be able to install this library... It is stuck at "select CRAN mirror" with no selection displayed. I wanted to use the data.tables read functionality since it offers better performance. The base R package produces the same error – Smajl Jan 20 '16 at 09:58
  • @Smaji Then, try to exclude that file and read all others. Then read the concerned file separately. i.e. files1 <- setdiff(files, "xyz.csv"); lapply(files1, fread)` – akrun Jan 20 '16 at 10:04
  • Could it be a path issue? (path different from working directory, so files are not found?) Does the issue persist if you use `full.names=T` inside list.files? – Heroka Jan 20 '16 at 10:04
  • @Heroka you seem to have just beaten me to the punch there. Although I would advise against `full.names=T` and rather suggest `full.names=TRUE`. It can cause problems using `T` as it can be used as a variable which could be set to `FALSE` whereas assignment to the name `TRUE` is prohibited. – jamieRowen Jan 20 '16 at 10:07

1 Answers1

12

I suspect the problem lies in the path to the files. Most likely because your working directory is one level up from the directory "test". Try:

    list.files(path = "test", pattern = ".csv", full.names = TRUE)

The full.names argument will include the path to the files in it's output.

jamieRowen
  • 1,509
  • 9
  • 14
  • Had the same issue, including `full.names` is a must if the files are not located in the working directory. – TheSciGuy May 02 '19 at 15:47