1

I have a folder with different csv files (more than 100) and every file is very large in size and takes a lifetime to open. Do you know how can I write a code just to see if the files opens correctly and if not which are the problematic files. I've tried with this but it was not working. 

library(data.table)
setwd("Working dir"
files<-list.files(pattern="*.csv")
numfiles <- length(files) 
for (i in c(1:numfiles)){  
    files[i] <- paste(".\\",files[i],sep="")  
    assign(gsub("[.]csv$","",files[i]),fread(files[i], header=FALSE))
}

Thank you for your help

Uwe
  • 41,420
  • 11
  • 90
  • 134
willpar
  • 35
  • 2
  • 1
    If you are just checking whether or not the file opens, why would you use `assign`? Further, why are you using `assign` at all? Don't do it. See this post to [save the data.frames to lists](http://stackoverflow.com/questions/17499013/how-do-i-make-a-list-of-data-frames) instead. If you are just reading them in to check, give each the same name. – lmo Jul 15 '16 at 12:28
  • 2
    `lapply(files, fread, header=FALSE)`, you can wrap it into `rbindlist` if you want it as a single table. – jangorecki Jul 15 '16 at 14:20
  • 1
    I'd do what Jan suggested. Note that (i) you do not need `.` to indicate the current dir when reading in files, since R will look there anyways and (ii) backslash dir indicators are OS-specific, so it's generally easier (and arguably more readable) to do `file.path(".", files[i])` – Frank Jul 15 '16 at 14:33
  • 1
    You say "but it was not working". Did you get an error message or an unexpected result? Please, be more precise when asking for help in debugging. Thank you. – Uwe Jul 16 '16 at 05:52
  • Thank you very much for your comments guys. The problem is that the files are badly formatted (different types of sep for examples), and fread gets confused as it cannot read the columns correctly. What I would like to do is try to skip the error (deleting the rows that give an error) and continue with the loop. How can I do that? Thanks! – willpar Jul 20 '16 at 12:19

0 Answers0