How to convert list of lists of key value pairs to dataframe?

Question

Note the input shape I am dealing with is not the same as in these similar questions: R list of lists to data.frame Converting a list of lists to a dataframe in R: The Tidyverse-way

I am working with a web api which returns results in this shape

listOfListsOfKeyVals <- lapply(1:5, function(i){
    list(
      col1 = i,
      col2 = runif(1)
    )
})

You might think that the following will work

do.call(rbind, listOfListsOfKeyVals)

But on closer inspection the result is actually a dataframe of lists

do.call(rbind, listOfListsOfKeyVals) |> tibble()

# A tibble: 5 × 1
  `do.call(rbind, listOfListsOfKeyVals)`[,"col1"] [,"col2"]
  <list>                                          <list>   
1 <int [1]>                                       <dbl [1]>
2 <int [1]>                                       <dbl [1]>
3 <int [1]>                                       <dbl [1]>
4 <int [1]>                                       <dbl [1]>
5 <int [1]>                                       <dbl [1]>

I have come up with the following solution

foreach(x = listOfListsOfKeyVals, .combine = rbind) %do% {
  as.data.frame(x)
} |> tibble()

But it is painfully slow for large data sets. Is there a better way?

You can call the rbind method for data frames directly - `do.call(rbind.data.frame, listOfListsOfKeyVals)`. `bind_rows()` as suggested in the answer below will most likely be faster though. — Ritchie Sacramento, Jun 08 '23 at 06:14

score 4 · Accepted Answer · answered Jun 08 '23 at 05:54

I think you are looking for dplyr::bind_rows.

library(dplyr)

set.seed(12)
listOfListsOfKeyVals <- lapply(1:5, function(i){
  list(
    col1 = i,
    col2 = runif(1)
  )
})

bind_rows(listOfListsOfKeyVals)

Output:

# A tibble: 5 × 2
   col1   col2
  <int>  <dbl>
1     1 0.0694
2     2 0.818 
3     3 0.943 
4     4 0.269 
5     5 0.169

score 1 · Answer 2 · answered Jun 08 '23 at 07:34

Another option is rrapply:

rrapply::rrapply(listOfListsOfKeyVals, how = "bind")

#   col1       col2
# 1    1 0.21794909
# 2    2 0.81600287
# 3    3 0.04631368
# 4    4 0.10518273
# 5    5 0.46489659

And a base R option with matrix:

l <- listOfListsOfKeyVals
data.frame(matrix(unlist(l), nrow = length(l), byrow = TRUE)) |>
  setNames(names(l[[1]]))

How to convert list of lists of key value pairs to dataframe?

2 Answers2