3

I want to filter my data if all of the values in a subset of columns are NA.

I found an answer here that works brilliantly for all columns, but in this case I want to exclude "wrapper" columns from the filter operation.

library(dplyr)

df <- tibble(a = letters[1:3], b = c(NA, NA, 3), c = c(NA, 2, 3), d = letters[1:3])

# works, but I've lost my surrounding columns

df %>%
  select(-one_of(c("a", "d"))) %>%
  filter_all(any_vars(!is.na(.)))

# only works if all columns are all not NA (or vice versa), I've lost my partially NA rows

df %>%
  filter(across(-one_of(c("a", "d")),
                ~ !is.na(.)))

Desired outcome:

> df[2:3,]
# A tibble: 2 x 4
  a         b     c d    
  <chr> <dbl> <dbl> <chr>
1 b        NA     2 b    
2 c         3     3 c 
Paul
  • 2,877
  • 1
  • 12
  • 28

3 Answers3

4

In the previous version of dplyr you could use filter_at in this way :

library(dplyr)

df %>% filter_at(vars(-one_of(c("a", "d"))), any_vars(!is.na(.)))

across do not have direct replacement of any_vars so one way would be to use Reduce here :

df %>% filter(Reduce(`|`, across(-one_of(c("a", "d")),~!is.na(.))))

# A tibble: 2 x 4
#  a         b     c d    
#  <chr> <dbl> <dbl> <chr>
#1 b        NA     2 b    
#2 c         3     3 c   
Ronak Shah
  • 377,200
  • 20
  • 156
  • 213
2

One option could be:

df %>%
 rowwise() %>%
 filter(sum(!is.na(c_across(-all_of(c("a", "d"))))) != 0)

  a         b     c d    
  <chr> <dbl> <dbl> <chr>
1 b        NA     2 b    
2 c         3     3 c  
tmfmnk
  • 38,881
  • 4
  • 47
  • 67
2

I found a similar example in vignette("colwise", package = "dplyr")

rowAny <- function(x) rowSums(x) > 0
df %>% filter(rowAny(across(-c(a,d), ~ !is.na(.))))
# A tibble: 2 x 4
  a         b     c d    
  <chr> <dbl> <dbl> <chr>
1 b        NA     2 b    
2 c         3     3 c  

But I would use this because it's a bit more readable:

rowAll <- function(x) rowSums(x) == length(x)
df %>% filter(!rowAll(across(-c(a,d), is.na)))
Edo
  • 7,567
  • 2
  • 9
  • 19