convert corpus into data.frame in R

Question

I'm using the tm package to apply stemming, and I need to convert the resulting data into a data frame. A solution for this can be found here R tm package vcorpus: Error in converting corpus to data frame, but in my case I have the content of the corpus as:

[[2195]]
i was very impress

instead of

[[2195]]
"i was very impress"

and because of this, if I apply

data.frame(text=unlist(sapply(mycorpus, `[`, "content")), stringsAsFactors=FALSE)

the result will be

<NA>.

Any help is much appreciated!

Code below as an example:

sentence <- c("a small thread was loose on the sandals, otherwise it looked good")
mycorpus <- Corpus(VectorSource(sentence))
mycorpus <- tm_map(mycorpus, stemDocument, language = "english")

inspect(mycorpus)

[[1]]
a small thread was loo on the sandals, otherwi it look good

data.frame(text=unlist(sapply(mycorpus, `[`, "content")), stringsAsFactors=FALSE)

 text
1 <NA>

Could you add some code with data to produce a minimal working example? From what I see so far, @RichardScriven 's idea sounds quite reasonable. — abel, Aug 25 '14 at 16:59
sure! by "can't manage" I meant that I wasn't able to apply the deparse function... — Cristina Cerqueira, Aug 25 '14 at 17:44
Oh, sorry. just try it on one element first. For example, `deparse(data[[2195]])` — Rich Scriven, Aug 25 '14 at 17:48
No, I'm sorry! :) I doesn't work also.. I think that the problem must be a different one, because in the example I'm giving at the end the input is a string vector... so it should work... any ideas? — Cristina Cerqueira, Aug 25 '14 at 17:52
I've solved! I've applied a workaround... If I apply gsub to the corpus, the result has class = character. It works for me. Thanks for your help! — Cristina Cerqueira, Aug 25 '14 at 18:06
@CristinaCerqueira - you should add that as an answer for future SO searchers. — Rich Scriven, Aug 25 '14 at 22:55

score 2 · Accepted Answer · answered Aug 25 '14 at 23:01

2

By applying

gsub("http\\w+", "", mycorpus)

the output has class = character, so it works in my case.

answered Aug 25 '14 at 23:01

Cristina Cerqueira

127
2
9

1

I would be helpful if you would write the complete code for the conversion. Otherwise readers are left to the hit and trial method. – Simone May 06 '16 at 19:55

score 1 · Answer 2 · answered Aug 25 '14 at 18:05

I'm unable to reproduce the problem using tm_0.6 in R 3.1.0 on a Mac:

> data.frame(text=unlist(sapply(mycorpus, `[`, "content")), stringsAsFactors=FALSE)
                                                                 text
content a small thread was loos on the sandals, otherwis it look good

If I had gotten those undesired results I would have immediately tried:

 data.frame(text=unlist(sapply(mycorpus, `[[`, "content")), stringsAsFactors=FALSE)

... reasoning that since 'constent' is a list-element name that [['content']] should have been able to so the serial extraction. It also looked to me that the unlist might not be needed with that approach:

> data.frame(text=sapply(mycorpus, `[[`, "content"), stringsAsFactors=FALSE)
                                                           text
1 a small thread was loos on the sandals, otherwis it look good

Thanks! If I apply your suggestion it gives me the error "Error in FUN(X[[1L]], ...) : subscript out of bounds"... But I've found a workaround for what I need. — Cristina Cerqueira, Aug 25 '14 at 18:11
I think you may have an outdated or broken installation of tm. — IRTFM, Aug 25 '14 at 19:02

convert corpus into data.frame in R

2 Answers2

Linked