I've to figure out the given statement is question or normal statement without defining any chunk grammar. I tried drawing a tree which needs grammer but it doesn't tell me whether it is a question or statement. Penn Treebank is one solution I've heard of but couldn't find any help for this
train_text = state_union.raw("text1.txt")
sample_text = state_union.raw("text2.txt")
custom_sent_tokenizer = PunktSentenceTokenizer(train_text)
#PunktSentenceTokenizer is an abtract class for sent_tokenizer()
tokenized = custom_sent_tokenizer.tokenize(sample_text)
##print (custom_sent_tokenizer)
print (tokenized)
try:
for i in tokenized:
words = nltk.word_tokenize(i)
tagged = nltk.pos_tag(words)
print tagged
chunkGram = r"""Chunk: {<RB.?>*<VB.?>*<NNP>+<NN>?}"""
chunkParser = nltk.RegexpParser(chunkGram)
chunked = chunkParser.parse(tagged)
print chunked
chunked.draw()
except Exception as e:
print(str(e))