Why this R list of lists fails?

Question

I have a list of .CSV filenames, which contents I want to read to memory as Array mA i.e. I want to put an array/list into the cell of mA so to make and access a list of lists or array of arrays.

data = read.csv("/home/masi/dataList.csv",header = FALSE,sep = ",") 
# P103C1.csv
# P103C2.csv 
# ...

i = 1
# http://stackoverflow.com/a/5688126/54964
mA <- vector("list", length(data))
for (d in data)
        data[i] = paste("/home/masi/Documents/CSV/", d, sep="") 
        # /home/masi/Documents/CSV/P103C1.csv
        # /home/masi/Documents/CSV/P103C2.csv
        # ...

        tmp = read.csv(data[i],header = TRUE,sep = ",")

        # http://stackoverflow.com/a/10776795/54964
        mA[i] = list(tmp[, 1]) # attempt to make a list of lists

        i = i + 1
end

Error

function (x, ...) 
UseMethod("end")
<bytecode: 0x28cb7f8>
<environment: namespace:stats>
function (x, ...) 
UseMethod("end")
<bytecode: 0x28cb7f8>
<environment: namespace:stats>

Code 2

data = scan(file = "/home/masi/dataList.csv", what = character()) # vector, access data[i]
files <- vector("list", length(data))

str(data)

for (i in 1:length(data)) {
        tmp = paste0("/home/masi/Documents/CSV/", data[i]) # faster than paste sep=""

        print(i)

        tmp # does not work

        tmp = read.csv(tmp, header = FALSE, sep=","); # no header with other approach
#       files[[i]] = list(tmp[, 1])
}

tmp # works if last tmp is commented out

dataList.csv

P103C1.csv
P103C2.csv
P111C1.csv
P111C2.csv
P118C1.csv

P103C1.csv

-0.04,-0.19
-0.045,-0.58
-0.04,-0.465
-0.035,-0.39
-0.01,-0.24

Output where error occur at i=23

Read 31 items
 chr [1:31] "P103C1.csv" "P103C2.csv" "P111C1.csv" "P111C2.csv" ...
...
[1] 21
[1] 22
[1] 23
Error in read.table(file = file, header = header, sep = sep, quote = quote,  : 
  no lines available in input
Calls: read.csv -> read.table
Execution halted

OS: Debian 8.5
R: 3.1.1
Linux kernel: 4.6 backports
Hardware: Asus Zenbook UX303UA

**a.** Arrays and lists are very different in R. Your code only has lists and data.frames (which are technically a type of list), not arrays. **b.** [Read the canonical answer about lists of data.frames](http://stackoverflow.com/a/24376207/4497050). — alistaire, Oct 30 '16 at 00:09

score 1 · Accepted Answer · edited May 23 '17 at 12:19

There three things going wrong here:

you were not writing a valid for loop. Have a look at Repetitive execution: for loops, repeat and while for the right syntax. Also, read ?Control;
To make a list of list, you need mA[[i]] <- list(tmp[, 1]), the double bracket. Have a read on The difference between [] and [[]] notations for accessing the elements of a list or dataframe for the difference. Also, read ?Extract;
Given your description, the "/home/masi/dataList.csv" is a file with a single column and 31 rows. Don't use read.csv to read it, otherwise you get a data frame with 1 column and 31 rows (then length(filelst) will be 1). You code really expect it to be a vector. Use scan (see below).

So the fix would be

filelst = scan(file = "/home/masi/dataList.csv", what = character()) 
i = 1
mA <- vector("list", length(filelst))
for (file in filelst) {
  tmp = paste0("/home/masi/Documents/CSV/", file)
  tmp = read.csv(tmp, header = TRUE)
  mA[[i]] = list(tmp[, 1])
  i = i + 1
  }

Follow-up

(I honestly think this should be a new question, regarding safe and robust code for reading files.)

Based on your update, some files maybe empty. To make the code more robust, we need try:

filelst = scan(file = "/home/masi/dataList.csv", what = character()) 
i = 1
mA <- vector("list", length(filelst))
for (file in filelst) {
  tmp = paste0("/home/masi/Documents/CSV/", file)
  test <- try(read.csv(tmp, header = TRUE), silent = TRUE)
  if (class(test) == "try-error") mA[[i]] = "Empty File; No Read!"
  else mA[[i]] = list(test[, 1])
  i = i + 1
  }

Have a read on ?try, and consider the following toy example for its use:

x <- ""
y <- "-0.04,-0.19
      -0.045,-0.58
      -0.04,-0.465
      -0.035,-0.39
      -0.01,-0.24"

tmp <- try(read.csv(text = x, header = FALSE), silent = TRUE)
tmp
#[1] "Error in read.table(file = file, header = header, sep = sep, quote = quote,  : \n  no lines available in input\n"
#attr(,"class")
#[1] "try-error"
#attr(,"condition")
#<simpleError in read.table(file = file, header = header, sep = sep, quote = quote,     dec = dec, fill = fill, comment.char = comment.char, ...): no lines available in input>

tmp <- try(read.csv(text = y, header = FALSE), silent = TRUE)
tmp
#      V1     V2
#1 -0.040 -0.190
#2 -0.045 -0.580
#3 -0.040 -0.465
#4 -0.035 -0.390
#5 -0.010 -0.240

I changed your if-else structure to be more explicit and stop immediately if a file is missing `if (class(test) == "try-error") { stop(sprintf("Empty file \"%d\".", i)); } else { files[[i]] = test; }`. — Léo Léopold Hertz 준영, Oct 30 '16 at 10:59

Why this R list of lists fails?

Code 2

1 Answers1