how to calculate correlation between one row and remaining row of a data frame

Question

I have a data like this

 name  col1  col2  col3
1    a 43.78 43.80 43.14
2    b 43.84 43.40 42.85
3    c 37.92 37.64 37.54
4    d 31.72 31.62 31.74

lets call it df

df<-structure(list(name = structure(1:4, .Label = c("a", "b", "c", 
"d"), class = "factor"), col1 = c(43.78, 43.84, 37.92, 31.72), 
    col2 = c(43.8, 43.4, 37.64, 31.62), col3 = c(43.14, 42.85, 
    37.54, 31.74)), class = "data.frame", row.names = c(NA, -4L
))

now I want to calculate the R2 and adjusted R2 between row d and the other rows

If I want to see all combinations, I can do the following for correlation

out <- cor(t(df[, -1]))
out[upper.tri(out, diag = TRUE)] <- NA
rownames(out) <- colnames(out) <- df$name
out <- na.omit(reshape::melt(t(out)))
out <- out[ order(out$X1, out$X2), ]

which gives me this

   X1 X2      value
5   a  b  0.8841255
9   a  c  0.6842705
13  a  d -0.6491118
10  b  c  0.9457125
14  b  d -0.2184630
15  c  d  0.1105508

but I only want between row d and the rest and also I want to have both correlation coefficient and adjusted R2

Answer 1

It's easier if you transpose your data frame first. After that use purrr::map and broom::tidy to get the job done

library(tidyverse)

df <- structure(list(name = structure(1:4, .Label = c("a", "b", "c", 
"d"), class = "factor"), col1 = c(43.78, 43.84, 37.92, 31.72), 
    col2 = c(43.8, 43.4, 37.64, 31.62), col3 = c(43.14, 42.85, 
    37.54, 31.74)), class = "data.frame", row.names = c(NA, -4L
))

# transpose df
df_transpose <- df %>% 
  gather(variable, value, -name) %>% 
  spread(name, value) %>% 
  select(-variable)

# loop through columns, apply `cor` vs 'd' column
colnames(df_transpose) %>%
  set_names() %>% 
  map(~ cor(df_transpose[, .x], df_transpose[, 'd'])) %>%
  map_dfr(., broom::tidy, .id = "var")

#> # A tibble: 4 x 2
#>   var        x
#>   <chr>  <dbl>
#> 1 a     -0.649
#> 2 b     -0.218
#> 3 c      0.111
#> 4 d      1

^{Created on 2019-03-15 by the reprex package (v0.2.1.9000)}

Answer 2

If I understand you right, you want the correlation between d and every single remaining column.

(M <- t(as.matrix(`rownames<-`(df1[-1], df$name))))
#          a     b     c     d
# col1 43.78 43.84 37.92 31.72
# col2 43.80 43.40 37.64 31.62
# col3 43.14 42.85 37.54 31.74

Due to vectorization we can calculate the correlation between d and the remainder very easily:

out <- t(cor(M[, 4], M[, -4]))

The R² is just the square of the correlation (Ref.) which we can cbind to the correlations.

`colnames<-`(cbind(out, out^2), c("cor", "r2"))
#          cor         r2
# a -0.6491118 0.42134617
# b -0.2184630 0.04772607
# c  0.1105508 0.01222148

 

(Note: Case you're wondering about the `colnames<-` form, you may want to read "Advanced R: 6.8.4 Replacement functions".)

---

Data

df1 <- structure(list(name = structure(1:4, .Label = c("a", "b", "c", 
"d"), class = "factor"), col1 = c(43.78, 43.84, 37.92, 31.72), 
    col2 = c(43.8, 43.4, 37.64, 31.62), col3 = c(43.14, 42.85, 
    37.54, 31.74)), class = "data.frame", row.names = c(NA, -4L
))