Classify similar words

Question

Im currently doing a textmining process in which I would like to transform similar words (table, tables etc...) into one word (table)'. I saw that the tm package offers a tool for this but this one does not support the language Im looking for. So therefore I want to create something myself.

For the function I want to have a link table ->

 a <- c("Table", "Tables", "Tree", "Trees")
 b <- c("Table", "Tree", "Chair", "Invoice")
 df <- data.frame(b, a)

So that i can automatically transfer all the "Tables" values into "Table"

Any thoughts on how I can do this?

If you're looking for an open source solution to do this type of thing, I strongly suggest using OpenRefine (formerly GoogleRefine) for an excellent and fast way for textmining. I know your question relates to R, but depending on how many variables you want to change and how big your dataset is, I do find this the most time-efficient solution. — OFish, Aug 07 '15 at 06:47

score 5 · Answer 1 · edited May 23 '17 at 12:21

Search for stemming in R, you can look here and you can try :

a <- c("Table", "Tables", "Tree", "Trees")
b <- c("Table", "Tree", "Chair", "Invoice")
library("SnowballC")
wordStem(words = a, language = "porter")
##[1] "Tabl" "Tabl" "Tree" "Tree"
library("tm") # tm use wordStem
stemCompletion(x = stemDocument(x = a), dictionary = b)
##   Tabl    Tabl    Tree    Tree 
##"Table" "Table"  "Tree"  "Tree"

Or more complicated to use but more complete, you can look at package korPus and use TreeTagger to process your text :

library("koRpus")
tagged.results <- treetag(tolower(a), treetagger="manual", format="obj",
                          TT.tknz=FALSE , lang="en",
                          TT.options=list(path="./TreeTagger", preset="en"))
tagged.results@TT.res
##   token tag lemma lttr wclass                   desc stop stem
##1  table  NN table    5   noun Noun, singular or mass   NA   NA
##2 tables NNS table    6   noun           Noun, plural   NA   NA
##3   tree  NN  tree    4   noun Noun, singular or mass   NA   NA
##4  trees NNS  tree    5   noun           Noun, plural   NA   NA

What you want is in :

tagged.results@TT.res$lemma
##[1] "table" "table" "tree"  "tree"

Classify similar words

1 Answers1