1

I want to subset my data (grades_overall) such that all rows which have a valued less than 30 in the column CSE are included in the new dataframe called lesst_than_30. 

> str(grades_overall)
'data.frame':   284 obs. of  20 variables:
 $ T1             : num  10 11 10 12 9 7 8 13 11 8 ...
 $ T2             : num  10 4 10 15 13 5 14 14 16 11 ...
 $ BS             : num  12 NA NA 13 9 6 16 3 8 10 ...
 $ BT             : num  9 8 13 12 11 3 7 14 12 13 ...
 $ BC             : num  5 4 14 14 12 6 11 7 12 13 ...
 $ BM             : num  1 NA 10 11 9 5 14 9 11 11 ...
 $ D1             : num  17 NA 14 14 11 9 11 15 15 17 ...
 $ D2             : num  15 12 15 10 16 17 10 16 11 16 ...
 $ D3             : num  16 6 8 10 12 6 7 19 12 13 ...
 $ D4             : num  11 NA 14 10 18 16 10 14 12 17 ...
 $ D5             : num  16 10 14 16 15 8 6 16 16 15 ...
 $ D6             : num  12 NA 15 12 11 11 10 17 16 17 ...
 $ Total_testscore: num  7 13 11 12 11 9 13 5 9 12 ...
 $ Programme      : Factor w/ 4 levels "ArchBrus","ArchGent",..: 1 2 2 2 2 2 2 3 4 2 ...
 $ Math_GPA       : Factor w/ 5 levels "<60%",">90%",..: 4 4 1 5 1 1 3 NA NA 4 ...
 $ ID             : num  1 3 6 7 8 12 13 14 15 16 ...
 $ SE_track       : Factor w/ 3 levels "ASO","KSO","TSO": 1 1 2 3 1 1 1 2 2 2 ...
 $ Gender         : Factor w/ 2 levels "male","female": 1 1 1 1 2 2 1 2 2 2 ...
 $ CSE            : num  83 33 67 100 67 17 50 83 83 83 ...
 $ Hours_Math_SE  : num  6 3 6 4 6 4 6 2 6 6 ...
> lesst_than_30 <- grades_overall[grades_overall[,19] <30,] #41
> head(lesst_than_30)
     T1 T2 BS BT BC BM D1 D2 D3 D4 D5 D6 Total_testscore Programme Math_GPA ID SE_track Gender CSE Hours_Math_SE
6     7  5  6  3  6  5  9 17  6 16  8 11               9  ArchGent     <60% 12      ASO female  17             4
NA   NA NA NA NA NA NA NA NA NA NA NA NA              NA      <NA>     <NA> NA     <NA>   <NA>  NA            NA
29    8 NA  9  8 NA NA  9 11  7 NA NA NA               9  ArchGent     <60% 48      ASO female  17             6
NA.1 NA NA NA NA NA NA NA NA NA NA NA NA              NA      <NA>     <NA> NA     <NA>   <NA>  NA            NA
NA.2 NA NA NA NA NA NA NA NA NA NA NA NA              NA      <NA>     <NA> NA     <NA>   <NA>  NA            NA
40    4 12  6  7 13  9  6  9 11 16  9 13              10  ArchGent     <NA> 66      ASO female  17             6

Where do the NA values come from ??

There are no observations where all the data is missing. How can this be fixed ?

Nneka
  • 1,764
  • 2
  • 15
  • 39
  • 1
    They come from the condition `grades_overall[,19] <30`. This is a logical index with `NA`'s wherever there are missing values in the data. Use `which(grades_overall[,19] <30)`. – Rui Barradas Aug 10 '19 at 11:58

2 Answers2

2

I do not know if this is a duplicate but an answer could be based on the following example.

x <- c(1, NA, 3)

x[x < 2]
#[1]  1 NA
x[which(x < 2)]
#[1] 1

What is happening is that the index has NA values in it and the result will also have them:

x < 2
#[1]  TRUE    NA FALSE
which(x < 2)
#[1] 1

This happens even if there are no NA's in the original data. The next example uses a logical index with one NA, the data is complete, but the result has a NA.

y <- c(1, 2, 3)
i <- c(TRUE, NA, FALSE)
y[i]
#[1]  1 NA

Not if the NAis removed with which.

y[which(i)]
#[1] 1
Rui Barradas
  • 70,273
  • 8
  • 34
  • 66
1

As @Rui mentioned grades_overall[,19] <30 return NA for values which are NA and when you subset them it returns NA rows. Consider an example using mtcars

df <- mtcars
df[1,1] <- NA

head(df[df[, 1] < 20, ])

#                   mpg cyl  disp  hp drat   wt  qsec vs am gear carb
#NA                  NA  NA    NA  NA   NA   NA    NA NA NA   NA   NA
#Hornet Sportabout 18.7   8 360.0 175 3.15 3.44 17.02  0  0    3    2
#Valiant           18.1   6 225.0 105 2.76 3.46 20.22  1  0    3    1
#Duster 360        14.3   8 360.0 245 3.21 3.57 15.84  0  0    3    4
#Merc 280          19.2   6 167.6 123 3.92 3.44 18.30  1  0    4    4
#Merc 280C         17.8   6 167.6 123 3.92 3.44 18.90  1  0    4    4

To fix this you can use another condition to check to check for NA values

df[df[, 1] < 20 & !is.na(df[, 1]), ]

Or use dplyr::filter which ignores NA values by default.

library(dplyr)
df %>%  filter(mpg < 20)
Ronak Shah
  • 377,200
  • 20
  • 156
  • 213