Filter by number of duplicates, in R

Question

I am utilising a dataset that includes one column that comprises of phone numbers, and one column that includes yes or no.

  phone.number yes.no
1   7539339393      Y
2    111111111      N
3   1234567890      Y
4   7539339393      N
5   1222222222      Y
6   3333333333      N
7   4444444444      Y
8   5555555555      N
9   7539339393      Y

dput below:

structure(list(phone.number = structure(c(7L, 1L, 3L, 7L, 2L, 
    4L, 5L, 6L, 7L), .Label = c("111111111", "1222222222", "1234567890", 
    "3333333333", "4444444444", "5555555555", "7539339393"), class = "factor"), 
        yes.no = structure(c(2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L), .Label = c("N", 
        "Y"), class = "factor")), .Names = c("phone.number", "yes.no"
    ), row.names = c(NA, -9L), class = "data.frame")

There are two things that I'd really appreciate some help with.

1. I am trying to create a subset that includes all phone numbers that have been repeated 3 or more times within the dataset. So, in the example provided, the number 7539339393 is repeated 3 times, the subset that I'd like to make would just include these observations and the corresponding yes and no values.

2. I am also trying to subset for all phone numbers that have returned yes and no. So, within the dataset provided, 7539339393 has corresponded to both yes and no, I'd like to create a subset with all of the phone numbers that return both and all of the corresponding yes and no values

Answer 1

In base R,

For 1:

df[df$phone.number %in% names(which(table(df$phone.number) >= 3)),]
#  phone.number yes.no
#1   7539339393      Y
#4   7539339393      N
#9   7539339393      Y

We subset based on phone numbers that appear 3 or more times. table takes a tally of all elements in the vector.

For 2:

df[as.logical(ave(as.character(df$yes.no), df$phone.number, FUN= function(x) all(c("Y", "N") %in% x))),]
#  phone.number yes.no
#1   7539339393      Y
#4   7539339393      N
#9   7539339393      Y

There are nice grouping functions in dplyr and data.table, this is one base R approach that may either highlight the strength of base R or the advantage of other packages depending on how you look at it. It is a bit verbose as a one liner. Working from the inside out, all is used to check if yes and no appear. The function ave helps to complete the operation by unique phone number.

Answer 2

Possible dplyr approaches..

For 1.:

df %>% group_by(phone.number) %>% filter(n() >= 3)
#Source: local data frame [3 x 2]
#Groups: phone.number [1]
#
#  phone.number yes.no
#        (fctr) (fctr)
#1   7539339393      Y
#2   7539339393      N
#3   7539339393      Y

(n() returns the number of rows per group in dplyr)

For 2.:

df %>% group_by(phone.number) %>% filter(all(c("Y", "N") %in% yes.no))
#Source: local data frame [3 x 2]
#Groups: phone.number [1]
#
#  phone.number yes.no
#        (fctr) (fctr)
#1   7539339393      Y
#2   7539339393      N
#3   7539339393      Y

For 1. + 2. combined:

df %>% group_by(phone.number) %>% filter(n() >= 3 & all(c("Y", "N") %in% yes.no))
#Source: local data frame [3 x 2]
#Groups: phone.number [1]
#
#  phone.number yes.no
#        (fctr) (fctr)
#1   7539339393      Y
#2   7539339393      N
#3   7539339393      Y

Answer 3

Here's a data.table solution, calling your dataset df:

library(data.table)
setDT(df)[,.SD[.N>2],by=phone.number]
#    phone.number yes.no
# 1:   7539339393      Y
# 2:   7539339393      N
# 3:   7539339393      Y

For the second part:

df[,.SD[length(unique(yes.no))>1], by=phone.number]

Note that setDT(df) converts df to a data.table, so you don't have to call it again in the second part.

This approach is likely to be faster than any other method, although unless your data set is extremely large you might not notice the difference.