R - Reading unknown number of files

Question

I'm new to R. I try to process some experimental data and I'm stuck with a bug when reading files. I want to read some data files in a folder but I don't know how many are there. I just know that all files are named "Subject1ManualX.log", X being 1 or higher. Since I didn't find any possibility to count how many files are in a folder, I try to open files in a while loop until an exception is raised (i.e. I try to open "Subject1Manual1.log", then "Subject1Manual2.log" etc).

Here is my code (prints are for debug):

# Script to work on ET data
temp <- 0

while (temp == 0){
  tryCatch({
    subjectData <- read.delim("D:/Doctorat/XPs/XP1-2_LIPSMWA/SmartEye/Subject1/Subject1Manual3.log")

  }, warning = function(w){
    print(paste("warning", i))
    temp <- 1

  }, error = function(e){
    print(paste("error", i))
    temp <- 1

  }, finally = {
    print("finished")
  })
}

Unfortunately it doesn't work (that's why I'm here...). I know I will have warnings and errors. If I handle warnings, R crashes because errors aren't handled ("all connection are in use" crash). If I only handle errors, it doesn't go through it and the while loop continues at each iteration.

Any idea on this matter would be highly appreciated (including better ways to read an unknown number of files). Thank you for your time!

Pyxel

EDIT: ok some nice persons answered how to import multiple data files, but for curiosity I would like to know how to deal with a try catch within a while loop. Any idea?

use `list.files` function to get all files, user `pattern` to select specific files... This can be used in for loops or what ever to process them easily..... — drmariod, Nov 10 '16 at 10:26

Gregory Demin · Accepted Answer · 2016-11-10T11:36:40.297

# Here we read path to all '*.log' files in folder
path_to_files = dir("D:/Doctorat/XPs/XP1-2_LIPSMWA/SmartEye/Subject1/", 
                    full.names = TRUE, 
                    recursive = TRUE, 
                    pattern = "\\.log$")

# further we read all files to list
files = lapply(path_to_files, read.delim)

# and finaly we combine files to single data.frame
# headers of files should be identical
res = do.call(rbind, files)

UPDATE Code with tryCatch and while. It is not safe and it grows data.frame in the loop - it is bad practice.

subjectData = TRUE
i = 1
res = NULL
while (!is.null(subjectData)){
    subjectData <-  tryCatch({
        read.delim(sprintf("D:/Doctorat/XPs/XP1-2_LIPSMWA/SmartEye/Subject1/Subject1Manual%s.log", i))

    }, warning = function(w){
        print(paste("warning", i))
        NULL

    }, error = function(e){
        print(paste("error", i))
        NULL

    })
    if(is.null(res)){
        res = subjectData
    } else {
        res = rbind(res, subjectData)
    }
    i = i + 1
}

Thank you, I think I used the wrong keywords to not find this "dir" function. I tried your solution and it works perfectly. Any idea though how to handle a trycatch in a while loop? — Pyxel, Nov 10 '16 at 11:17
> Thank you so much for your help! I won't use it since your first solution is definitely cleaner, but I was curious on how to solve my initial problem. — Pyxel, Nov 10 '16 at 11:58

R - Reading unknown number of files

1 Answers1