How to flatten a list of lists?

Question

The tm package extends c so that, if given a set of PlainTextDocuments it automatically creates a Corpus. Unfortunately, it appears that each PlainTextDocument must be specified separately.

e.g. if I had:

foolist <- list(a, b, c); # where a,b,c are PlainTextDocument objects

I'd do this to get a Corpus:

foocorpus <- c(foolist[[1]], foolist[[2]], foolist[[3]]);

I have a list of lists of 'PlainTextDocuments that looks like this:

> str(sectioned)
List of 154
 $ :List of 6
  ..$ :Classes 'PlainTextDocument', 'TextDocument', 'character'  atomic [1:1] Developing assessment models   Developing models
  .. .. ..- attr(*, "Author")= chr "John Smith"
  .. .. ..- attr(*, "DateTimeStamp")= POSIXlt[1:1], format: "2013-04-30 12:03:49"
  .. .. ..- attr(*, "Description")= chr(0) 
  .. .. ..- attr(*, "Heading")= chr "Research Focus"
  .. .. ..- attr(*, "ID")= chr(0) 
  .. .. ..- attr(*, "Language")= chr(0) 
  .. .. ..- attr(*, "LocalMetaData")=List of 4
  .. .. .. ..$ foo           : chr "bar"
  .. .. .. ..$ classification: chr "Technician"
  .. .. .. ..$ team          : chr ""
  .. .. .. ..$ supervisor    : chr "Bill Jones"
  .. .. ..- attr(*, "Origin")= chr "Smith-John_e.txt"

#etc., all sublists have 6 elements

So, to get all my PlainTextDocuments into a Corpus, this would work:

sectioned.Corpus <- c(sectioned[[1]][[1]], sectioned[[1]][[2]], ..., sectioned[[154]][[6]])

Can anyone suggest an easier way, please?

ETA: foo<-unlist(foolist, recursive=FALSE) produces a flat list of PlainTextDocuments, which still leaves me with the problem of feeding a list element by element to c

score 108 · Accepted Answer · edited Jan 07 '19 at 11:39

108

I expect that unlist(foolist) will help you. It has an option recursive which is TRUE by default.

So unlist(foolist, recursive = FALSE) will return the list of the documents, and then you can combine them by:

do.call(c, unlist(foolist, recursive=FALSE))

do.call just applies the function c to the elements of the obtained list

edited Jan 07 '19 at 11:39

zx8754

52,746
12
114
209

answered Apr 30 '13 at 12:58

DrDom

4,033
1
21
23

1

Also consider using `NCmisc::Unlist()` for unlisting beyond the first level. – Megatron Nov 20 '18 at 19:34
@zx8754 for the very similar large list, after importing a json file, I'm applying the same code, but I get the error "unlist arguments imply differing number of rows: 1, 0", which I understand, but i thought the code deals with lists of different levels. Any ideas? Beginner R user here... Thanks in advance for any help! – choabf Feb 18 '22 at 15:42
@choabf, I advise you to create a new question, where put an example of your data and code you used. – DrDom Feb 19 '22 at 16:11

Michael · Answer 2 · 2017-03-24T01:37:28.137

34

Here's a more general solution for when lists are nested multiple times and the amount of nesting differs between elements of the lists:

 flattenlist <- function(x){  
  morelists <- sapply(x, function(xprime) class(xprime)[1]=="list")
  out <- c(x[!morelists], unlist(x[morelists], recursive=FALSE))
  if(sum(morelists)){ 
    Recall(out)
  }else{
    return(out)
  }
}

edited Mar 24 '17 at 01:37

answered Jan 26 '17 at 20:58

Michael

5,808
4
30
39

4

just to make this a bit more comprehensible I'll just point out that identifying lists using class(xprime)[1]=="list") is necessary (rather than using is.list) when your nested objects are of classes that inherit from lists (i.e. note that is.list(data.frame(3)) evaluates to TRUE) – Michael Mar 24 '17 at 01:33
2

also note that this doesn't retain the order of the original structure – Michael Mar 24 '17 at 01:35
Michael, how would you go about retaining the order of the original structure? – Mihai Aug 12 '19 at 12:43
1

probably just replace `out <- c(` with an `mapply` statement that takes `x` and `morelists` as arguments then unlists only for elements where `morelists` is TRUE – Michael Mar 19 '21 at 16:00
1

or replace the first two lines with a single `lapply` that combines testing and unlisting – Michael Mar 19 '21 at 16:00
2

Very useful: My usecase is `map(some_nested_list, flattenlist) %>% bind_rows()` to produce a tibble. – shosaco Dec 16 '21 at 08:46

Zeus · Answer 3 · 2021-05-30T08:23:53.217

12

Here's another method that worked for my list of lists.

df <- as.data.frame(do.call(rbind, lapply(foolist, as.data.frame)))

Or take a look at new functions in tidyr which work well.

rectangle a nested list into a tidy tibble

rectangling

    lst <-  list(
      list(
        age = 23,
        gender = "Male",
        city = "Sydney"
      ),
      list(
        age = 21,
        gender = "Female",
        city = "Cairns"
      )
    )
      
    tib <- tibble(lst)  %>% 
      unnest_wider(lst)

df <- as.data.frame(tib)

edited May 30 '21 at 08:23

answered Sep 24 '20 at 05:22

Zeus

1,496
2
24
53

Thanks for sharing. ```unnest_wider``` from the tidyr package worked perfectly. The other code you posted gave me an error message "Error in match.names(clabs, names(xi)) : names do not match previous names" --> The data I am working with is downloaded as a JSON from Facebook. It is highly nested. Not all lists in the list are equally long. The data structure also varies between downloaded files. – Simone Nov 03 '21 at 09:25
1

@Simone, I think you should post a new question with a simple replication of your data, then some can try to help you. SO doesn't like questions within questions – Zeus Nov 08 '21 at 00:51
I [did](https://stackoverflow.com/posts/69838172/revisions). Wanted to point out that there are "normally" nested lists and highly nested lists. For the latter the tidyr package is useful. --> hence the comment and not a new question. – Simone Nov 08 '21 at 09:30

How to flatten a list of lists?

3 Answers3

Linked

Related