0

This question is related to How can I replace a factor levels with the top n levels (by some metric), plus [other]?. As a metric I want to use the number of occurrences of the factor. I know I can do it by making a list of the occurrences, but I was wondering if there is a prettier way.

Example:

library(data.table);
library(plyr);
fac <- data.table(score = as.factor(c(3,4,5,3,3,3,5)));
ocCnt <- data.table(lapply(fac,count)$score);
fac$occurrence <- 0;
for(i in 1:length(fac$score)){fac$occurrence[i]<-ocCnt[x==fac$score[i]]$freq};

Then I could use the function described in the referenced question/answer:

hotfactor= function(fac,by,n=10,o="other") {
   levels(fac)[rank(-xtabs(by~fac))[levels(fac)]>n] <- o
   fac
}

To continue the example, if we want only to see the most popular factor we do:

hotfactor(fac$score,fac$occurrence,1);

To get the answer:

[1] 3 other other 3 3 3 other

Levels: 3 other

So my question is, can I do this without having to add a list which counts the occurrences?

Note that I want to do this for the n most popular factors (not just for the most popular factor).

Community
  • 1
  • 1
Nieke Aerts
  • 209
  • 2
  • 11

2 Answers2

1

Use table and which.max:

score <- factor(c(3,4,5,3,3,3,5))
levels(score)[- which.max(table(score))] <- "other"
#[1] 3     other other 3     3     3     other
#Levels: 3 other

Obviously this breaks ties by taking the first maximum value.

If you want to keep the top two levels:

score <- factor(c(3, 4,5,3,3,3,5), levels =c(4,3,5))

levels(score)[!levels(score) %in% names(sort(table(score), decreasing = TRUE)[1:2])] <- "other"
#[1] 3     other 5     3     3     3     5    
#Levels: other 3 5
Roland
  • 127,288
  • 10
  • 191
  • 288
0

If you don't know how many levels you need to group say, 90% of your data and are willing to use dplyr, you could do something along the following lines:

library(dplyr)

df <- data.frame(
        f = factor(mapply(rep, letters[1:5], 2^(1:5)) %>% unlist(use.names = F))
)

df %>% 
    count(f, sort = T) %>% 
    mutate(p = cumsum(n) / nrow(df))

#      A tibble: 5 x 3
#        f     n         p
#   <fctr> <int>     <dbl>
# 1      e    32 0.5161290
# 2      d    16 0.7741935
# 3      c     8 0.9032258
# 4      b     4 0.9677419
# 5      a     2 1.0000000

(top <- df %>% 
    count(f, sort = T) %>% 
    mutate(p = cumsum(n) / nrow(df)) %>%
    filter(cumall(p < .91)) %>% 
    select(f) %>% 
    unlist(use.names = F))

# [1] e d c
# Levels: a b c d e

levels(df$f) <- factor(c(levels(df$f), 'z'))
df$f[!df$f %in% top] <- 'z'

df %>% 
    count(f, sort = T) %>% 
    mutate(p = cumsum(n) / nrow(df))

#  A tibble: 4 x 3
#        f     n         p
#   <fctr> <int>     <dbl>
# 1      e    32 0.5161290
# 2      d    16 0.7741935
# 3      c     8 0.9032258
# 4      z     6 1.0000000
sbaldrich
  • 1,062
  • 12
  • 15