13

I need to parse sentences from a paragraph in Python. Is there an existing package to do this, or should I be trying to use regex here?

hippietrail
  • 15,848
  • 18
  • 99
  • 158
David542
  • 104,438
  • 178
  • 489
  • 842
  • Are there double-spaces after the end of each sentence? – Zach Young Feb 28 '12 at 00:07
  • 2
    Your problem statement doesn't provide sufficient information for us to work with. – RanRag Feb 28 '12 at 00:10
  • 2
    There are some answers here: http://stackoverflow.com/questions/116494/python-regular-expression-to-split-paragraphs – martineg Feb 28 '12 at 00:10
  • 4
    "Purely syntactic approaches using regexps sound problematic... just think of the 5.5 ways that Prof. Smith from the U.S. told us periods can be used." – DSM Feb 28 '12 at 00:17
  • These things are usually done by dedicated sentence splitter tools / library modules. Trying to do with regexes alone is not going to produce good results. The better splitters have been trained. – tchrist Feb 28 '12 at 00:33

2 Answers2

46

The nltk.tokenize module is designed for this and handles edge cases. For example:

>>> from nltk import tokenize
>>> p = "Good morning Dr. Adams. The patient is waiting for you in room number 3."
>>> tokenize.sent_tokenize(p)
['Good morning Dr. Adams.', 'The patient is waiting for you in room number 3.']
strcat
  • 5,376
  • 1
  • 27
  • 31
0

Here is how I am getting the first n sentences:

def get_first_n_sentence(text, n):
    endsentence = ".?!"
    sentences = itertools.groupby(text, lambda x: any(x.endswith(punct) for punct in endsentence))
    for number,(truth, sentence) in enumerate(sentences):
        if truth:
            first_n_sentences = previous+''.join(sentence).replace('\n',' ')
        previous = ''.join(sentence)
        if number>=2*n: break #

    return first_n_sentences

Reference: http://www.daniweb.com/software-development/python/threads/303844

David542
  • 104,438
  • 178
  • 489
  • 842
  • it will not work if the text contains URL's, or something with punctuation (period) on the term like, Ms., Dr., etc. – mannysz Aug 03 '16 at 00:20