Why do "subset" and "[" on a dataframe give slightly different results?

Question

Could someone explain me why I get different results in my last two lines of code (identical() calls) below? These two objects seem to be identical objects, but when I use them in an apply function, I get some trouble:

df <- data.frame(a = 1:5, b = 6:2, c = rep(7,5))
df_ab <- df[,c(1,2)]
df_AB <- subset(df, select = c(1,2))
identical(df_ab,df_AB)
[1] TRUE

apply(df_ab,2,function(x) identical(1:5,x))
    a     b 
TRUE FALSE

apply(df_AB,2,function(x) identical(1:5,x))
    a     b 
FALSE FALSE

Answer 1

The apply() function coerces its first argument to a matrix before calling the function on each column. So your data frames are coerced to matrix objects. A consequence of that conversion is that as.matrix(df_AB) has non-null rownames, while as.matrix(df_ab) does not:

> str(as.matrix(df_ab))
 int [1:5, 1:2] 1 2 3 4 5 6 5 4 3 2
 - attr(*, "dimnames")=List of 2
  ..$ : NULL
  ..$ : chr [1:2] "a" "b"
> str(as.matrix(df_AB))
 int [1:5, 1:2] 1 2 3 4 5 6 5 4 3 2
 - attr(*, "dimnames")=List of 2
  ..$ : chr [1:5] "1" "2" "3" "4" ...
  ..$ : chr [1:2] "a" "b"

So when you apply() subset a column of df_AB, you get a named vector, which is not identical to an unnamed vector.

apply(df_AB, 2, str)
 Named int [1:5] 1 2 3 4 5
 - attr(*, "names")= chr [1:5] "1" "2" "3" "4" ...
 Named int [1:5] 6 5 4 3 2
 - attr(*, "names")= chr [1:5] "1" "2" "3" "4" ...
NULL

Contrast that with the subset() function, which selects rows using a logical vector for the value of i. And it looks like subsetting a data.frame with a non-missing value for i causes this difference in the row.names attribute:

> str(as.matrix(df[1:5, 1:2]))
 int [1:5, 1:2] 1 2 3 4 5 6 5 4 3 2
 - attr(*, "dimnames")=List of 2
  ..$ : chr [1:5] "1" "2" "3" "4" ...
  ..$ : chr [1:2] "a" "b"
> str(as.matrix(df[, 1:2]))
 int [1:5, 1:2] 1 2 3 4 5 6 5 4 3 2
 - attr(*, "dimnames")=List of 2
  ..$ : NULL
  ..$ : chr [1:2] "a" "b"

You can see the all the gory details of the difference between the data.frames using the .Internal(inspect(x)) function. You can look at those yourself, if you're interested.

As Roland pointed out in his comments, you can use the .row_names_info() function to see the differences in only the row names.

Notice that when i is missing, the result of .row_names_info() is negative, but it is positive if you subset with a non-missing i.

> .row_names_info(df_ab, type=1)
[1] -5
> .row_names_info(df_AB, type=1)
[1] 5

What these values mean is explained in ?.row_names_info:

type: integer.  Currently ‘type = 0’ returns the internal
      ‘"row.names"’ attribute (possibly ‘NULL’), ‘type = 2’ the
      number of rows implied by the attribute, and ‘type = 1’ the
      latter with a negative sign for ‘automatic’ row names.

Answer 2

If you want to compare the values 1:5 with the values in the columns, you should not use apply since apply transforms the data frames to matrices before the functions are applied. Due to the row names in the subset created with [ (see @Joshua Ulrich's answer), the values 1:5 are not identical to a named vector including the same values.

You should instead use sapply to apply the identical function to the columns. This avoids transforming the data frames to matrices:

> sapply(df_ab, identical, 1:5)
    a     b 
 TRUE FALSE 
> sapply(df_AB, identical, 1:5)
    a     b 
 TRUE FALSE 

As you can see, in both data frames the values in the first column are identical to 1:5.

Answer 3

In one version (using [) your columns are integers, while in the other version (using subset) your columns are named integers.

apply(df_ab, 2, str)

 int [1:5] 1 2 3 4 5
 int [1:5] 6 5 4 3 2
NULL


apply(df_AB, 2, str)

 Named int [1:5] 1 2 3 4 5
 - attr(*, "names")= chr [1:5] "1" "2" "3" "4" ...
 Named int [1:5] 6 5 4 3 2
 - attr(*, "names")= chr [1:5] "1" "2" "3" "4" ...
NULL

Answer 4

Looking at the structure of those two object s before they get submitted to apply shows only one difference: in the rownames, but not a difference that I would have expected to produce the difference you are seeing. I do not see Joshua's current offer of 'subset' as logical indexing as explaining this. Why row.names = c(NA, -5L)) produces a named result when extracting with "[" is as yet unexplained.

> dput(df_AB)
structure(list(a = 1:5, b = c(6L, 5L, 4L, 3L, 2L)), .Names = c("a", 
"b"), row.names = c(NA, 5L), class = "data.frame")
> dput(df_ab)
structure(list(a = 1:5, b = c(6L, 5L, 4L, 3L, 2L)), .Names = c("a", 
"b"), class = "data.frame", row.names = c(NA, -5L))

I do agree that it is the as.matrix coercion which needs further investigation:

> attributes(df_AB[,1])
NULL
> attributes(df_ab[,1])
NULL
> attributes(as.matrix(df_AB)[,1])
$names
[1] "1" "2" "3" "4" "5"