How to extract all matching patterns (words in a string) in a dataframe column?

Question

I have two dataframes. one (txt.df) has a column with a text I want to extract phrases from (text). The other (wrd.df) has a column with the phrases (phrase). both are big dataframes with complex texts and strings but lets say:

txt.df <- data.frame(id = c(1, 2, 3, 4, 5),
                     text = c("they love cats and dogs", "he is drinking juice", 
                              "the child is having a nap on the bed", "they jump on the bed and break it",
                              "the cat is sleeping on the bed"))


wrd.df <- data.frame(label = c('a', 'b', 'c', 'd', 'e', 'd'),
                     phrase = c("love cats", "love dogs", "juice drinking", "nap on the bed", "break the bed",
                              "sleeping on the bed"))

what I finally need is a txt.df with another column which contains labels of the phrases detected.

what I tried was creating a column in wrd.df in which I tokenized the phrases like this

wrd.df$token <- sapply(wrd.df$phrase, function(x) unlist(strsplit(x, split = " ")))

and then tried to write a custom function to sapply over the tokens column with grepl/str_detect get the names (labels) of those which were all true

Extract.Fun <- function(text, df, label, token){
  for (i in token) {
  truefalse[i] <- sapply(token[i], function (x) grepl(x, text))
  truenames[i] <- names(which(truefalse[i] == T))
  removedup[i] <- unique(truenames[i])
  return(removedup)
}

and then sapply this custom function on my txt.df$text to have a new column with the labels.

txt.df$extract <- sapply(txt.df$text, function (x) Extract.Fun(x, wrd.df, "label", "token"))

but I'm not good with custom functions and am really stuck. I would appreciate any help. P.S. It would be very good if i could also have partial matches like "drink juice" and "broke the bed"... but it's not a priority... fine with the original ones.

Answer 1

If you need to match the exact phrases, the regex_join() from the fuzzyjoin-package is what you need.

fuzzyjoin::regex_join( txt.df, wrd.df, by = c(text = "phrase"), mode = "left" )

  id                                 text label              phrase
1  1              they love cats and dogs     a           love cats
2  2                 he is drinking juice  <NA>                <NA>
3  3 the child is having a nap on the bed     d      nap on the bed
4  4    they jump on the bed and break it  <NA>                <NA>
5  5       the cat is sleeping on the bed     d sleeping on the bed

If you want to match all words, I guess you can build a regex out of the phrases that cover such behaviour...

update

#build regex for phrases
#done by splitting the phrases to individual words, and then paste the regex together
wrd.df$regex <- unlist( lapply( lapply( strsplit( wrd.df$phrase, " "), 
                                        function(x) paste0( "(?=.*", x, ")", collapse = "" ) ),
                                function(x) paste0( "^", x, ".*$") ) )


fuzzyjoin::regex_join( txt.df, wrd.df, by = c(text = "regex"), mode = "left" )

  id                                 text label              phrase                                        regex
1  1              they love cats and dogs     a           love cats                     ^(?=.*love)(?=.*cats).*$
2  1              they love cats and dogs     b           love dogs                     ^(?=.*love)(?=.*dogs).*$
3  2                 he is drinking juice     c      juice drinking                ^(?=.*juice)(?=.*drinking).*$
4  3 the child is having a nap on the bed     d      nap on the bed      ^(?=.*nap)(?=.*on)(?=.*the)(?=.*bed).*$
5  4    they jump on the bed and break it     e       break the bed            ^(?=.*break)(?=.*the)(?=.*bed).*$
6  5       the cat is sleeping on the bed     d sleeping on the bed ^(?=.*sleeping)(?=.*on)(?=.*the)(?=.*bed).*$