0

I am trying to create a new data frame TopWords from an existing one. The original data frame data_to_export has too much many words(bios), and I would like to only keep words (bios)that were used frequently, but I also need to keep the ID numbers associated with each word.

This is what I've come up with, but it doesn't work. It doesn't like the if conditional statement, but I don't know how else to do it.

TopWords<- data_to_export if freq_terms(data_to_export$bios2 > 4)

I would like to end up with the same data from data_to_export, but only the data for cases that have words that occur fives times or more.

For example, 

data_to_export (original data)
ID  bios2
1    i
1    love
1    playing
1    soccer
2    i
2    am
2    a
2    teacher
2    mom
2    grandma
2    sister
3    i
3    think
3    soccer
3    is
3    the 
3    best
4    soccer
4    player
5    i
5    like
5    soccer
5    i
5    could
5    play
5    soccer
5    all
5    day


New data frame:
1   i
1   soccer
2   i
3   i
3   soccer
4   soccer
5   i
5   soccer
5   i
5   soccer

Any help would be greatly appreciated. Thanks!

lwe
  • 323
  • 1
  • 8
  • It's easier to help you if you include a simple [reproducible example](https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) with sample input and desired output that can be used to test and verify possible solutions. – MrFlick Sep 16 '19 at 20:24
  • I've tried to add an example above now. Thanks! – lwe Sep 16 '19 at 20:36
  • Where does `freq_terms` come from? That line of code isn't standard R syntax—I don't know of situations where you could just put the name of the data frame and then go right into an if statement like that – camille Sep 16 '19 at 21:26
  • Using `dplyr`, you can do `data_to_export %>% group_by(bios2) %>% filter(n() > 4) ` – Ronak Shah Sep 17 '19 at 00:41
  • camile, it's from the the ```qdap``` package – lwe Sep 17 '19 at 13:34

1 Answers1

0

Edit: Oddly enough, that actually doesn't work. Two instances of i was not picked up by the string comparison, but I'm unable to explain why that is.

Exchange the last line for this instead:

TopWords <- data_to_export[grep(paste0('\\b', names(freq)[freq], '\\b', collapse = '\\b|\\b'), data_to_export$bios2), ]

Here's a method using base R:

data_to_export <- data.frame(
  ID = c(rep(1, 4), rep(2, 7), rep(3, 6), rep(4, 2), rep(5, 9)),
  bios2 = c(
    'i', 'love', 'playing', 'soccer', 'i', 'am', 'a', 'teacher', 'mom',
    'grandma', 'sister', 'i', 'think', 'soccer', 'is', 'the', 'best', 'soccer',
    'player', 'i', 'like', 'soccer', 'i', 'could', 'play', 'soccer', 'all',
    'day'
  )
)
freq <- table(data_to_export$bios2) >= 5
TopWords <- subset(data_to_export, bios2 == names(freq)[freq])

table counts the frequency of string values and subset filters the data.frame for the desired strings with 5 or more occurrences.

Benjamin Ye
  • 508
  • 2
  • 7