Why would my residuals for a linear model in R have one less value than the data set I used to create it?

Question

I created a linear model in R using county data from the ACS. There are 3140 entries in my data set, and they all have their corresponding fips codes. I'm trying to make a map of the residuals from my linear model, but I only have 3139 residuals. Does anyone know if there's something R does when creating a linear model that is responsible for this, and how I can fix it so that I can create this map? Thanks!

In response to the suggestion of checking for NAs, I ran this:

which(completedata$fipscode == NA)
integer(0)

R code if it helps:

sectorcodes <- read.csv("sectorcodes1.csv") #ruralubrancode, median hh income
sectorcodesdf <- data.frame(sectorcodes)
religion <- read.csv("Religion2.csv")
religiondf <- data.frame(religion)

merge1 <- merge(sectorcodesdf,religiondf, by = c('fipscode'))
merge1df <- data.frame(merge1)

family <- read.csv("censusdataavgfamsize.csv") #avgfamilysize
familydf <- data.frame(family)

merge2 <- merge(merge1df, familydf, by = c('fipscode'))
merge2df <- data.frame(merge2)

gradrate <- read.csv("censusdatahsgrad.csv")
gradratedf <- data.frame(gradrate)
evenmoredata2 <- merge(gradrate,merge2df, by=c("fipscode"))

#write.csv(evenmoredata2, file = "completedataset.csv")
completedata <- read.csv("completedataset.csv")
completedatadf <- data.frame(completedata)

lm8 <- lm(completedatadf$hsgrad ~ completedatadf$averagefamilysize*completedatadf$Rural_urban_continuum_code_2013*completedatadf$TOTADH*completedatadf$Median_Household_Income_2016)
summary(lm8)

library(blscrapeR)
library(RgoogleMaps)
library(choroplethr)
library(acs)
attach(acs)
require(choroplethr)

dataframe1 <- data.frame(completedatadf$fipscode,completedatadf$averagefamilysize)
names(dataframe1) <- c("region","value")

dataframe2 <- data.frame(completedata$fipscode,completedata$hsgrad)
names(dataframe2) <- c("region","value")

residdf <- data.frame(lm8$residuals)
dataframe3 <- data.frame(completedata$fipscode,lm8$residuals)
names(dataframe3) <- c("region","value")
county_choropleth(dataframe1)
county_choropleth(dataframe2)
county_choropleth(dataframe3)

When I try to run dataframe3, the error message is:

dataframe3 <- data.frame(completedata$fipscode,lm8$residuals)
Error in data.frame(completedata$fipscode, lm8$residuals) : 
arguments imply differing number of rows: 3140, 3139

When asking for help, you should include a simple [reproducible example](https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) with sample input and desired output that can be used to test and verify possible solutions. How exactly did you create `lm8`? Do you have missing values in your data? — MrFlick, May 07 '18 at 18:15
I added the rest of my R code and also checked for NAs using which() — jax, May 07 '18 at 18:24

score 1 · Answer 1 · answered May 07 '18 at 18:19

This can be caused by an NA in the response. For example, using the builtin BOD data frame note that there are 5 residuals in this example but 6 rows in b:

b <- BOD
b[3, 2] <- NA
nrow(b)
## [1] 6
fm <- lm(demand ~ Time, b)
resid(fm)
##          1          2          4          5          6 
## -0.3578947 -0.2657895  1.6184211 -0.6894737 -0.3052632

We can handle that by specifying na.action = na.exclude when running lm. Note that now there are 6 residuals with the extra one being NA.

fm <- lm(demand ~ Time, b, na.action = na.exclude)
resid(fm)
##          1          2          3          4          5          6 
## -0.3578947 -0.2657895         NA  1.6184211 -0.6894737 -0.3052632

na.exclude did not fix the error message, however putting resid(lm8) instead of lm8$residuals seems to have fixed the problem, so thank you for writing it like that! — jax, May 07 '18 at 22:44

score 0 · Answer 2 · answered May 07 '18 at 18:14

0

Try

data.frame(na.omit(completedata$fipscode), lm8$residuals)

Probably, you're data has NA values.

answered May 07 '18 at 18:14

lcgodoy

763
5
14

I got the same error: Error in data.frame(na.omit(completedata$fipscode), lm8$residuals) : arguments imply differing number of rows: 3140, 3139 – jax May 07 '18 at 18:19

Why would my residuals for a linear model in R have one less value than the data set I used to create it?

2 Answers2