Why pivot_longer() nests simply four integer values in its output

Question

I was wondering why my pivot_longer() call below returns nested output under the value column?

foo <- function(){ 
 n_subj = 5
 n_trials = 20
 subj_intercepts = rnorm(n_subj, 0, 1)
 slope = .6
 mx = 30
 data = data.frame(subject = rep(1:n_subj, each=n_trials),
                   intercept = rep(subj_intercepts, each=n_trials)) %>%
   mutate(x = rnorm(n(), mx, 1),
          y = intercept + slope*(x-mean(x)) + rnorm(n(), 0, 1))
 
mlm = coef(summary(lmer(y ~ x + (1|subject), data=data)))[2,1]
ols = coef(summary(lm(y ~ x, data=data)))[2,1]
list(ols=ols, mlm=mlm)
}
 
kk <- data.frame(t(replicate(2, foo())))

pivot_longer(kk, everything())

#  name  value    
#  <chr> <list>   
#1 ols   <dbl [1]>
#2 mlm   <dbl [1]>
#3 ols   <dbl [1]>
#4 mlm   <dbl [1]>

score 5 · Accepted Answer · answered Sep 09 '21 at 22:29

5

You are creating a list in your function: Add/Wrap unlist at the end of your function code to get a tibble:

foo <- function(){ 
    n_subj = 5
    n_trials = 20
    subj_intercepts = rnorm(n_subj, 0, 1)
    slope = .6
    mx = 30
    data = data.frame(subject = rep(1:n_subj, each=n_trials),
                      intercept = rep(subj_intercepts, each=n_trials)) %>%
        mutate(x = rnorm(n(), mx, 1),
               y = intercept + slope*(x-mean(x)) + rnorm(n(), 0, 1))
    
    mlm = coef(summary(lmer(y ~ x + (1|subject), data=data)))[2,1]
    ols = coef(summary(lm(y ~ x, data=data)))[2,1]
    unlist(list(ols=ols, mlm=mlm))
}

output

  name  value
  <chr> <dbl>
1 ols   0.600
2 mlm   0.581
3 ols   0.441
4 mlm   0.528

answered Sep 09 '21 at 22:29

TarJae

72,363
6
19
66

1

I think `unlist(list(ols=ols, mlm=mlm))` is a little bit cumbersome. Won't `c(ols = ols, mlm = mlm)` be the same? Nevertheless: Nice solution. :-) – Martin Gal Sep 09 '21 at 22:38
Maybe something like this: `pivot_longer(kk, everything()) %>% mutate(id = rep(row_number(), each=2, length.out = n()))` – TarJae Sep 09 '21 at 22:38
1

@TarJae Imho it's easier the other way round: `mutate(rn = row_number()) %>% pivot_longer(-rn)`. – Martin Gal Sep 09 '21 at 22:40
@Reza: If you want your function to output a list, then you could use this code to get a dataframe afterwards with `unnest` : `pivot_longer(kk, everything()) %>% unnest(cols = c(value)) %>% mutate(id = rep(row_number(), each=2, length.out = n()))` – TarJae Sep 09 '21 at 22:40
@Martin Gal. Yes indeed thanks for pointing there!!! – TarJae Sep 09 '21 at 22:42

score 4 · Answer 2 · answered Sep 09 '21 at 22:26

Let's take a closer look at kk:

kk <- data.frame(t(replicate(2, foo())))

returns

        ols       mlm
1 0.6450264 0.6830734
2 0.6443189 0.6523727

But if we transform it into a tibble, which gives us more information

tibble(kk)

we see the data.frame actually contains lists.

# A tibble: 2 x 2
  ols       mlm      
  <list>    <list>   
1 <dbl [1]> <dbl [1]>
2 <dbl [1]> <dbl [1]>

So pivot_longer, which converts the data.frame into a tibble, just shows you the "real" form of your data.

Extracting an element of kk gives you the same result:

kk[1,1]
#> [[1]]
#> [1] 0.6450264

@Reza I don't have knowledge of an `.id` argument of `rerun()`. But if you are going to use `purrr`, you could use `map_df(2, ~rerun(.x, foo()), .id = "name")`. The `map_df*`-function uses `.id`. — Martin Gal, Sep 09 '21 at 22:54

score 1 · Answer 3 · answered Sep 10 '21 at 01:29

I would suggest to return a tibble/data.frame at the end of the function instead of list.

library(dplyr)
library(purrr)

foo <- function(){ 
    #...
    #...
    tibble(ols=ols, mlm=mlm)
}

We can then replicate it with simplify = FALSE and bind the output with bind_rows.

kk <- bind_rows(replicate(2, foo(), simplify = FALSE))
kk

# A tibble: 2 x 2
#    ols   mlm
#  <dbl> <dbl>
#1 0.862 0.681
#2 0.685 0.581

pivot_longer(kk, everything())

#  name  value
#  <chr> <dbl>
#1 ols   0.862
#2 mlm   0.681
#3 ols   0.685
#4 mlm   0.581

Why pivot_longer() nests simply four integer values in its output

3 Answers3

Linked