How to code remove NA in "with" base in R

Question

My simple data set look like the following.

+--------+------+--------+-----+-----+
|  Time  | Firm |  Out   | Lab | Cap |
+--------+------+--------+-----+-----+
| Jan-00 | A    | 161521 | 261 | 13  |
| Feb-00 | A    | 142452 | 334 | 15  |
| Mar-00 | A    | 365697 | 156 | 14  |
| Apr-00 | A    | 355789 | 134 | 12  |
| May-00 | A    | 376843 | 159 | 15  |
| Jun-00 | A    | 258762 | 119 | 12  |
| Jul-00 | A    | 255447 | 41  | 45  |
| Aug-00 | A    | 188545 | 247 | 75  |
| Sep-00 | A    | 213663 | 251 | NA  |
| Oct-00 | A    | 273209 | 62  | 12  |
| Nov-00 | A    | 317468 | 525 | 15  |
| Dec-00 | A    | 238668 | 217 | 16  |
| Jan-01 | B    | 241286 | 298 | 42  |
| Feb-01 | B    | 135288 | 109 | 45  |
| Mar-01 | B    | 363609 | 7   | 24  |
| Apr-01 | B    | 318472 | NA  | 56  |
| May-01 | B    | 446279 | 0   | 12  |
| Jun-01 | B    | 390230 | 50  | 12  |
| Jul-01 | B    | 118945 | 143 | 45  |
| Aug-01 | B    | 174887 | 85  | NA  |
| Sep-01 | B    | 183770 | 80  | 15  |
| Oct-01 | B    | 197832 | 214 | 12  |
| Nov-01 | B    | 317468 | 525 | 15  |
| Dec-01 | B    | 238668 | 217 | 16  |
+--------+------+--------+-----+-----+

The above data set can be reproduced using the following code.

    structure(list(Time = structure(c(9L, 7L, 15L, 1L, 17L, 13L, 
    11L, 3L, 23L, 21L, 19L, 5L, 10L, 8L, 16L, 2L, 18L, 14L, 12L, 
    4L, 24L, 22L, 20L, 6L), .Label = c("Apr-00", "Apr-01", "Aug-00", 
    "Aug-01", "Dec-00", "Dec-01", "Feb-00", "Feb-01", "Jan-00", "Jan-01", 
    "Jul-00", "Jul-01", "Jun-00", "Jun-01", "Mar-00", "Mar-01", "May-00", 
    "May-01", "Nov-00", "Nov-01", "Oct-00", "Oct-01", "Sep-00", "Sep-01"
    ), class = "factor"), Firm = structure(c(1L, 1L, 1L, 1L, 1L, 
    1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
    2L, 2L, 2L), .Label = c("A", "B"), class = "factor"), Out = c(161521L, 
    142452L, 365697L, 355789L, 376843L, 258762L, 255447L, 188545L, 
    213663L, 273209L, 317468L, 238668L, 241286L, 135288L, 363609L, 
    318472L, 446279L, 390230L, 118945L, 174887L, 183770L, 197832L, 
    317468L, 238668L), Lab = c(261L, 334L, 156L, 134L, 159L, 119L, 
    41L, 247L, 251L, 62L, 525L, 217L, 298L, 109L, 7L, NA, 0L, 50L, 
    143L, 85L, 80L, 214L, 525L, 217L), Cap = c(13L, 15L, 14L, 12L, 
    15L, 12L, 45L, 75L, NA, 12L, 15L, 16L, 42L, 45L, 24L, 56L, 12L, 
    12L, 45L, NA, 15L, 12L, 15L, 16L)), .Names = c("Time", "Firm", 
    "Out", "Lab", "Cap"), class = "data.frame", row.names = c(NA, 
    -24L))

First I get the panel structure for the panel data regression using the following command

    library(zoo)
    library(plm)

    Sys.setlocale("LC_TIME", "English")
    dat["time1"] <- as.yearmon(dat$Time,format="%b-%y")
    pdat <-pdata.frame(dat,index=c("Firm","time1"))

Now run the regression

    Model1<- plm(Out ~ Lab+Cap+I(0.5*(Lab^2))
                +I(0.5*(Cap^2))+I(Lab*Cap),data=pdat)
    summary(Model1)

Then short cuts are created for the resulted estimates as below.

    a1 <- coef( Model1 )[ "Lab" ]
    a2 <- coef( Model1 )["Cap"]
    a11 <- coef( Model1  )[ "I(0.5*(Lab^2))" ]
    b22 <- coef( Model1  )[ "I(0.5*(Cap^2))" ]
    a12 <-a21<- coef( Model1  )[ "I(Lab*Cap)" ]

    pdat$mpLab <- with( pdat,+ a1 + a11 * a1 + a12*a2)

I have this code to calculate marginal product of area. But my data frame has got some missing values and I need to include the code to remove those missing values from the above calculation. Otherwise the answer also comes as NA. I just tried to include below at the end of above code.

    pdat$mpLab <- with( pdat,+ a1 + a11 * a1 + a12*a2,na.rm=TRUE)

But it does not take that tag.The dataframe pdat is a panel data set analysed using panel data regression. a1, a2,a11,a12 are coefficients of the particular regression. pdat has some missing data and I believe this pdat$mpLab gives NA for the each observation due to that missing data in the pdat.

Much appreciated if anybody can help me.

It would be great if you could supply a minimal reproducible example to go along with your question. Something we can work from and use to show you how it might be possible to answer your question. That way others can also befit form your question, and the accompanying answer, in the future. You can have a look at [this SO post](https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) on how to make a great reproducible example in R. — Eric Fail, Jan 13 '16 at 22:52
You could try `rowSums(with(pdat, cbind(a1, a11*a1, a12*a2)), na.rm = TRUE)` — talat, Jan 13 '16 at 22:59
@Marat Talipov _ no I want to keep the missing values as it is and not to replace by zero. — Doo, Jan 14 '16 at 04:16

score 3 · Answer 1 · answered Jan 13 '16 at 23:03

You are using + to compute rowwise sums and for that there's a special function rowSums which also has a na.rm = argument. I use cbind create a matrix of the columns you want to sum and then apply rowSums(..., na.rm = TRUE), so it could look like this:

pdat$earea <- rowSums(with(pdat, cbind(a1, a11*a1, a12*a2)), na.rm = TRUE)

Here's an example with some fake data:

set.seed(1)
x <- data.frame(a1 = c(1:4, NA), a11 = sample(5), a12 = c(NA, 2:5), a2 = sample(5))
x$check <- rowSums(with(x, cbind(a1, a11*a1, a12*a2)), na.rm = TRUE) 
x
#  a1 a11 a12 a2 check
#1  1   2  NA  5     3
#2  2   5   2  4    20
#3  3   4   3  2    21
#4  4   3   4  3    28
#5 NA   1   5  1     5

@ docendo discimus I edited my question with the codes and the data set. — Doo, Jan 14 '16 at 04:16

score 0 · Accepted Answer · answered Jan 14 '16 at 19:26

0

I'm not completely sure this is the problem you are referring to, but this is definitely a problem in the code you submitted : some indices of model coefficients where not correctly typed, they are missing a few space characters. When the indices don't exist, NAs are returned.

When I used the correct typing:

a1 <- coef( Model1 )[ "Lab" ]
a2 <- coef( Model1 )["Cap"]
a11 <- coef( Model1  )[ "I(0.5 * (Lab^2))" ]
b22 <- coef( Model1  )[ "I(0.5 * (Cap^2))" ]
a12 <-a21<- coef( Model1  )[ "I(Lab * Cap)" ]

I did get all coefficients having values, and

pdat$mpLab <- with( pdat,+ a1 + a11 * a1 + a12*a2)

did populate the dataframe.

Hope this is what you want

answered Jan 14 '16 at 19:26

HubertL

19,246
3
32
51

Yes it's working with your coding correction for my simple data set, but still not with my big data set – Doo Jan 15 '16 at 01:15
@Doo then you'll have to make a test dataset that reproduces – HubertL Jan 15 '16 at 01:19
I figured out. That was a problem with space in the code as you indicated. – Doo Jan 15 '16 at 03:46

How to code remove NA in "with" base in R

2 Answers2