2

I am trying to split a paragraph into sentences but it is not working. I feel like this should be an easy thing, and like I must be making a stupid mistake. I am getting it to work with string split, but want to figure out the regex.

Example:

lorem <- "Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum."

strsplit(lorem, "[.]") 

[1] "Lorem Ipsum is simply dummy text of the printing and typesetting industry"                                                                                                                                      
[2] " Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book"                                     
[3] " It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged"                                                                                       
[4] " It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum"

But when I use regex: 

grep("[^\\.\\!\\?]*[\\.\\!\\?]", lorem, value=TRUE, perl=TRUE )

[1] "Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum."

It just pops out the original input

Jamie Leigh
  • 359
  • 1
  • 4
  • 18
  • 3
    `grep` doesn't split into sentences. It just checks whether a pattern is present or not in the whole string. – akrun Dec 06 '16 at 02:27
  • You really need a parser here. Even splitting the string on period won't always work, e.g. with words like `Mr.` and abbreviations. – Tim Biegeleisen Dec 06 '16 at 02:29
  • You can't use grep, you could use `regmatches` or similar to extract the sentences. http://stackoverflow.com/questions/19720144/r-string-removes-punctuation-on-split/19720181#19720181 – hwnd Dec 06 '16 at 02:41

2 Answers2

1

We can use package qdap

library(qdap)
sent_detect(lorem)

Output:

[1] "Lorem Ipsum is simply dummy text of the printing and typesetting industry."
[2] "Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book."
[3] "It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged."
[4] "It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum."

MFR
  • 2,049
  • 3
  • 29
  • 53
0

As @akrun said, grep only checks whether a pattern is present or not in the whole string.

To do your task, we can use str_match_all in stringr package, which extracts matched groups from a string.

unlist( stringr::str_match_all(lorem, "[^\\s][^\\.\\!\\?]+[\\.\\!\\?]{1}") )

Output:

[1] "Lorem Ipsum is simply dummy text of the printing and typesetting industry."
[2] "Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book."
[3] "It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged."
[4] "It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum."

pe-perry
  • 2,591
  • 2
  • 22
  • 33