R: Append Object to an Array: Why is Data Structure not Preserved?

Question

When I append an object to a vector or list, the structure of the appended object somehow gets mixed up. In my case I want to append the predict()result of a glm()model which is itself a named list (according to the docs):

If se.fit = TRUE, a list with components

fit: Predictions, as for se.fit = FALSE.

se.fit: Estimated standard errors.

residual.scale: A scalar giving the square root of the dispersion used in computing the standard errors.

Example setup:

dat <- data.frame(x=c(1,2,3,4,5), param=c(1,2,1,2,1), y=c(3,6,5,8,8))
mdl <- glm(y ~ x + param, data=dat)
lvls <- c(1,2)
x <- seq(1,5, length.out=10)

I've tried several approaches

# 1st approach
pred1 <- c()
for(i in 1:length(lvls)) {
    prd <- predict(mdl, data.frame(x=x, param=lvls[i]), se.fit=TRUE)
    pred1 <- c(pred1, prd)
}    
print(attributes(prd))  # this returns what I expect
# $names
# [1] "fit"            "se.fit"         "residual.scale"
print(attributes(pred1[1]))
# $names
# [1] "fit"



# 2nd approach
pred2 <- list()
for(i in 1:length(lvls)) {
    prd <- predict(mdl, data.frame(x=x, param=lvls[i]), se.fit=TRUE)
    pred2[i] <- prd  # this raises an error 
}



# 3rd approach
pred3 <- list()
for(i in 1:length(lvls)) {
    prd <- predict(mdl, data.frame(x=x, param=lvls[i]), se.fit=TRUE)
    pred3 <- append(pred3, prd)
}
print(attributes(pred3[1]))
# $names
# [1] "fit"

As you can see, only the first named attribute $fit remains in the vector or list. I can't access $se.fit like I would expect:

pred3[1]$se.fit  # I would expect to be able to do this for the first prediction
# NULL

Instead, it seems that all named attributes of the original object were flattened into a sequence with a single attribute for each item

pred3[1]
# $fit
#        1        2        3        4        5        6        7        8        9       10 
# 2.933333 3.466667 4.000000 4.533333 5.066667 5.600000 6.133333 6.666667 7.200000 7.733333 

pred3[2]
# $se.fit
#         1         2         3         4         5         6         7         8         9        10 
# 0.3126944 0.2769618 0.2467901 0.2244334 0.2123744 0.2123744 0.2244334 0.2467901 0.2769618 0.3126944 

pred3[3]
# $residual.scale
# [1] 0.3651484

pred3[4]
# $fit
#        1        2        3        4        5        6        7        8        9       10 
# 4.600000 5.133333 5.666667 6.200000 6.733333 7.266667 7.800000 8.333333 8.866667 9.400000 

pred3[5]
# $se.fit
#         1         2         3         4         5         6         7         8         9        10 
# 0.3464102 0.3145315 0.2883185 0.2694301 0.2594708 0.2594708 0.2694301 0.2883185 0.3145315 0.3464102 

pred3[6]
# $residual.scale
# [1] 0.3651484

1. Why does this happen?
2. How can I append the object 'as is'

Answer 1

how about this

library('magrittr')
pred1 <- c()
for(i in 1:length(lvls)) {
  prd <- predict(mdl, data.frame(x=x, param=lvls[i]), se.fit=TRUE)
  pred1 %<>% append(prd)
}

Answer 2

I have found an answer. I've just discovered the double bracket indexing [[

This thread explains the difference. The crucial thing:

For lists, one generally uses [[ to select any single element, whereas [ returns a list of the selected elements.

I've played around with this and came to the solution:

# 4th approach - like 2nd but [[ instead [
pred4 <- list()
for(i in 1:length(lvls)) {
    prd <- predict(mdl, data.frame(x=x, param=lvls[i]), se.fit=TRUE)
    pred4[[i]] <- prd
}


pred4[[1]]$fit
#        1        2        3        4        5        6        7        8        9       10 
# 2.933333 3.466667 4.000000 4.533333 5.066667 5.600000 6.133333 6.666667 7.200000 7.733333 

pred4[[1]]$se.fit
#         1         2         3         4         5         6         7         8         9        10 
# 0.3126944 0.2769618 0.2467901 0.2244334 0.2123744 0.2123744 0.2244334 0.2467901 0.2769618 0.3126944 

pred4[[1]]$residual.scale
# [1] 0.3651484

I think I also understand why my 1st approach doesn't work. The explanation is here (?Extract)

Extract or Replace Parts of an Object

Description

Operators acting on vectors, matrices, arrays and lists to extract or replace parts. Usage

x[i]
x[i, j, ... , drop = TRUE]
x[[i, exact = TRUE]]
x[[i, j, ..., exact = TRUE]]
x$name
getElement(object, name)

So the named entries of the predict-result-list are on the same level or index dimension where the vector does its append operation. The extract / indexing methods are equivalent. That means the $names are just aliases for the [[index]]. And so with single brackets I appended not the elements, but one list of 3 named elements to another.