Just convert your corpus into a dataframe and use regular expressions to detect the sentences.
Here is a function that uses regular expressions to detect sentences in a paragraph and returns each individual sentence.
chunk_into_sentences <- function(text) {
break_points <- c(1, as.numeric(gregexpr('[[:alnum:] ][.!?]', text)[[1]]) + 1)
sentences <- NULL
for(i in 1:length(break_points)) {
res <- substr(text, break_points[i], break_points[i+1])
if(i>1) { sentences[i] <- sub('. ', '', res) } else { sentences[i] <- res }
}
sentences <- sentences[sentences=!is.na(sentences)]
return(sentences)
}
...Using one paragraph inside a corpus from the tm package.
text <- paste('Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.')
mycorpus <- VCorpus(VectorSource(text))
corpus_frame <- data.frame(text=unlist(sapply(mycorpus, `[`, "content")), stringsAsFactors=F)
Use as follows:
chunk_into_sentences(corpus_frame)
Which gives us:
[1] "Lorem Ipsum is simply dummy text of the printing and typesetting industry."
[2] "Lorem Ipsum has been the industry standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book."
[3] "It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged."
[4] "It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum."
Now with a larger corpus
text1 <- "Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industrys standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum."
text2 <- "It is a long established fact that a reader will be distracted by the readable content of a page when looking at its layout. The point of using Lorem Ipsum is that it has a more-or-less normal distribution of letters, as opposed to using 'Content here, content here', making it look like readable English. Many desktop publishing packages and web page editors now use Lorem Ipsum as their default model text, and a search for 'lorem ipsum' will uncover many web sites still in their infancy. Various versions have evolved over the years, sometimes by accident, sometimes on purpose (injected humour and the like)."
text3 <- "There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which looks reasonable. The generated Lorem Ipsum is therefore always free from repetition, injected humour, or non-characteristic words etc."
text_list <- list(text1, text2, text3)
my_big_corpus <- VCorpus(VectorSource(text_list))
Use as follows:
lapply(my_big_corpus, chunk_into_sentences)
Which gives us:
$`1`
[1] "Lorem Ipsum is simply dummy text of the printing and typesetting industry."
[2] "Lorem Ipsum has been the industrys standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book."
[3] "It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged."
[4] "It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum."
$`2`
[1] "It is a long established fact that a reader will be distracted by the readable content of a page when looking at its layout."
[2] "The point of using Lorem Ipsum is that it has a more-or-less normal distribution of letters, as opposed to using 'Content here, content here', making it look like readable English."
[3] "Many desktop publishing packages and web page editors now use Lorem Ipsum as their default model text, and a search for 'lorem ipsum' will uncover many web sites still in their infancy."
$`3`
[1] "There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable."
[2] "If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text."
[3] "All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet."
[4] "It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which looks reasonable."
[5] "The generated Lorem Ipsum is therefore always free from repetition, injected humour, or non-characteristic words etc."
