convert nested list with varying dimension into data frame

Question

I have a nested list that I'm trying to convert to a data frame. I've spent hours checking solutions on here and nothing has worked for me.

Here's the code I'm using to generate this example:

#library('devtools')
#install_github('blockspring/blockspring.R')
library(blockspring)
library(rjson)

comp <- blockspringRunParsed("alternative-to-search", list( 
  "query" = "Qualtrics", 
  "page_number" = NULL 
), list(
  "api_key" = "br_2884_cc148ba53e81b7b3eeaf1ac65bc539cf62fe9850"))$params

This creates a list called "comp" with nested lists in my environment

 > str(comp)
    List of 1
     $ results:List of 25
      ..$ :List of 6
    .. ..$ icon       : chr "http://cdn.altrn.tv/icons/google-forms_53250.png?width=32&height=32&mode=crop&anchor=middlecenter"
      .. ..$ likes      : chr "29"
      .. ..$ name       : chr "Google Drive - Forms"
      .. ..$ pageLink   : chr "http://alternativeto.net/software/google-forms/"
      .. ..$ projectType: chr "Free"
      .. ..$ description: chr "Google Forms, part of Google Drive , is a tool...
    ...several more results returned.....
    ..$ :List of 6
          .. ..$ icon       : chr "http://cdn.altrn.tv/icons/webform_19004.png?width=32&height=32&mode=crop&anchor=middlecenter"
          .. ..$ likes      : chr "2"
          .. ..$ name       : chr "Webform"
          .. ..$ pageLink   : chr "http://alternativeto.net/software/webform/"
          .. ..$ projectType: chr "Freemium"
          .. ..$ description: chr "Free surveys that can go with you anywhere.    Build  ------  Make surveys crazy-fast with the Webform builder. Multiple pages,"| __truncated__
          ..$ :List of 5
          .. ..$ likes      : chr "1"
          .. ..$ name       : chr "Factile"
          .. ..$ pageLink   : chr "http://alternativeto.net/software/factile/"
          .. ..$ projectType: chr "Free"
          .. ..$ description: chr "Factile is free online survey building

One thing I tried was the rbindlist function with the data.table package

> rbindlist(comp, use.names=T, fill=T)

But I received this error:

Error in rbindlist(comp, use.names = T, fill = T) : 
  fill=TRUE, but names of input list at position 1 is NULL. All items of input list must have names set when fill=TRUE.

I also tried this from the qdap package:

> head(mtabulate(lapply(comp, `[[`,"results")))

It returned:

data frame with 0 columns and 1 row
Warning messages:
1: In is.na(x) : is.na() applied to non-(list or vector) of type 'NULL'
2: In is.na(x) : is.na() applied to non-(list or vector) of type 'NULL'

When I try to convert it to a data frame it creates a one row df. Ideally, the dataframe would have 25 rows with 6 columns. There are some which are missing the "$icon" item, so for those with a missing value an "NA" should be inputted.

Any help would be fantastic! Thanks

> sessionInfo()
R version 3.1.3 (2015-03-09)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 8 x64 (build 9200)

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] qdap_2.2.0             RColorBrewer_1.1-2     qdapTools_1.1.0       
 [4] qdapRegex_0.2.0        qdapDictionaries_1.0.2 ggplot2_1.0.0         
 [7] data.table_1.9.4       plyr_1.8.1             rjson_0.2.15          
[10] blockspring_0.4       

loaded via a namespace (and not attached):
 [1] assertthat_0.1      bitops_1.0-6        chron_2.3-45       
 [4] colorspace_1.2-5    DBI_0.3.1           devtools_1.7.0     
 [7] digest_0.6.8        dplyr_0.4.1         gdata_2.13.3       
[10] gender_0.4.3        grid_3.1.3          gridExtra_0.9.1    
[13] gtable_0.1.2        gtools_3.4.1        httr_0.6.1         
[16] igraph_0.7.1        jsonlite_0.9.14     magrittr_1.5       
[19] MASS_7.3-39         mime_0.2            munsell_0.4.2      
[22] NLP_0.1-6           openNLP_0.2-4       openNLPdata_1.5.3-1
[25] parallel_3.1.3      plotrix_3.5-11      proto_0.3-10       
[28] R.methodsS3_1.7.0   R.oo_1.19.0         R.utils_2.0.2      
[31] Rcpp_0.11.5         RCurl_1.95-4.5      reports_0.1.4      
[34] reshape2_1.4.1      rJava_0.9-6         scales_0.2.4       
[37] slam_0.1-32         stringr_0.6.2       tcltk_3.1.3        
[40] tm_0.6              tools_3.1.3         venneuler_1.1-0    
[43] wordcloud_2.5       xlsx_0.5.7          xlsxjars_0.6.1     
[46] XML_3.98-1.1

It would help to include a [reproducible example](http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) The data you shared is not in a reproducible format. Also, be as precise as possible about the desired output for the sample input. — MrFlick, May 27 '15 at 03:19

score 0 · Answer 1 · 2015-05-27T15:28:05.283

0

You can use rbind.fill from plyr library

library(plyr)
do.call(rbind.fill, lapply(unlist(comp, rec=F), as.data.frame))

edited May 27 '15 at 15:28

answered May 27 '15 at 03:24

The first line creates a df with one row and 148 variables (which is not right). The second line gives an error "Error in rbind(deparse.level, ...) : numbers of columns of arguments do not match" – cheevahagadog May 27 '15 at 03:51
2

you haven't provided reproducible data to try different solutions, so how do you expect someone to give you the desired output? @NathanCheever – Pierre L May 27 '15 at 05:12
1

I've added my code to generate the output so I hope this helps. – cheevahagadog May 27 '15 at 13:04
@NathanCheever Did you try the third option? It seems to work for that example you added. – May 27 '15 at 15:27
I'm not totally sure what you mean by the "third option". Does that mean the one answer submitted? (that is giving me an error saying: Error: All inputs to rbind.fill must be data.frames) – cheevahagadog May 28 '15 at 06:26

convert nested list with varying dimension into data frame

1 Answers1