Extract Nouns from Text (Java)

Question

Does anyone know the easiest way to extract only nouns from a body of text?

I've heard about the TreeTagger tool and I tried giving it a shot but couldn't get it to work for some reason.

Any suggestions?

Thanks Phil

EDIT:

 import org.annolab.tt4j.*; 
TreeTaggerWrapper tt = new TreeTaggerWrapper(); 

try { tt.setModel("/Nouns/english.par"); 

tt.setHandler(new TokenHandler() { 
     void token(String token, String pos, String lemma) {    
     System.out.println(token+"\t"+pos+"\t"+lemma); } }); 
     tt.process(words); // words = list of words 

     } finally { tt.destroy(); 
} 

That is my code, English is the language. I was getting the error : The type new TokenHandler(){} must implement the inherited abstract method TokenHandler.token. Am I doing something wrong?

Answer 1

First you will have to tokenize your text. This may seem trivial (split at any whitespace may work for you) but formally it is harder. Then you have to decide what is a noun. Does "the car park" contain one noun (car park), two nouns (car, park) or one noun (park) and one adjective (car)? This is a hard problem, but again you may be able to get by without it.

Does "I saw the xyzzy" identify a noun not in a dictionary? The word "the" probably identifies xyzzy as a noun.

Where are the nouns in "time flies like an arrow". Compare with "fruit flies like a banana" (thanks to Groucho Marx).

We use the Brown tagger (Java) (http://en.wikipedia.org/wiki/Brown_Corpus) in the OpenNLP toolkit (opennlp.tools.lang.english.PosTagger; opennlp.tools.postag.POSDictionary on http://opennlp.sourceforge.net/) to find nouns in normal English and I'd recommend starting with that - it does most of your thinking for you. Otherwise look at any of the POSTaggers (http://en.wikipedia.org/wiki/POS_tagger) or (http://www-nlp.stanford.edu/links/statnlp.html#Taggers).

In part-of-speech tagging by computer, it is typical to distinguish from 50 to 150 separate parts of speech for English, for example, NN for singular common nouns, NNS for plural common nouns, NP for singular proper nouns (see the POS tags used in the Brown Corpus)

There is a very full list of NLP toolkits in http://en.wikipedia.org/wiki/Natural_language_processing_toolkits. I would strongly suggest you use one of those rather than trying to match against Wordnet or other collections.

Answer 2

my following code works with TreeTagger:

public List<String> tag(String str) {
    final List<String> tagLemme = new ArrayList<String>();
    String[] tokens =tokenizer.tokenize(str);
      System.setProperty("treetagger.home", "parametresTreeTagger/TreeTagger");
    TreeTaggerWrapper tt = new TreeTaggerWrapper<String>();
    try {
        tt.setModel("parametresTreeTagger/english/english.par");
        tt.setHandler(new TokenHandler<String>(){
                public void token(String token, String pos, String lemma) {
                        tagLemme.add(token + "_" + pos + "_" + lemma);
                        //System.out.println(token + "_" + pos + "_" + lemma);
                }
        });
        tt.process(asList(tokens));
     } catch (IOException e) {
        e.printStackTrace();
      } catch (TreeTaggerException e) {
        e.printStackTrace();
    }
finally {
        tt.destroy();
}
    return tagLemme;
}

Answer 3

Check out LingPipe. This can supposedly pick out named entities from English text. But I must confess that NLP isn't my area of expertise.

Answer 4

Based on your edit:

The error says that you must override the abstract method token, and you have a definition for token in your anonymous inner class, but maybe the signature of your token-override doesn't match the signature of the abstract method defined in TokenHandler?

Answer 5

Have a look at the WordNet database. This lexical database. You could try matching each word against it and check if it's a noun.

I doubt that you will have 100% precision, though; the database doesn't have a match for every possible word in the english language, but at least it's a start.

Answer 6

Easiest way would probably be to compare each word in the text with a dictionary of nouns. After that you're probably going to have to do some elementary parsing and accept approximate correctness in the results. Lots of online references to parsing natural languages.

Answer 7

Find a dictionary web site with an API (e.g. WS, RESTful) which you can use to run search queries against.

The results should come in an easily consumable format (e.g. XML, JSON) and should of course include the word's lexical category.