How to get the confidence intervals for LOWESS fit using R?

Question

I didn't find any satisfactory answer to the confidence intervals (CIs) for LOWESS regression line of the 'stats' package of R:

plot(cars, main = "lowess(cars)")
lines(lowess(cars), col = 2)

But I'm unsure how to draw a 95% CI around it?? However, I know I could get the estimated variance from

V = s^2*sum(w^2)

where, s2= estimated error variance, and w=weights applied to the X. Therefore, the 95% CIs should be

Y plus/minus 2*sqrt(V(Y))

I know there's a way of getting the CIs from loess fit, but I'd rather prefer LOWESS because it is robust. Thanks for your suggestions.

https://stat.ethz.ch/pipermail/r-help/2008-August/170011.html — rawr, Mar 28 '14 at 16:22
Oh, right. Good point @rawr. Do you want the confidence bands or confidence ellipse? — Rich Scriven, Mar 28 '14 at 16:22
Why do you say `loess` isn't robust? `?loess.control` for example documents the number of iterations to use in robust fitting. If you want SEs from a LO(W)ESS fit, I'd probably use `loess()` and `predict.loess()`. — Gavin Simpson, Mar 28 '14 at 16:37

Henry David Thorough · Accepted Answer · 2014-03-28T17:07:36.303

34

You can do this with predict() and loess(). lowess is older than loess and has fewer features, though it is a bit faster. But in this context, I'd use loess as follows.

plot(cars)
plx<-predict(loess(cars$dist ~ cars$speed), se=T)

lines(cars$speed,plx$fit)
lines(cars$speed,plx$fit - qt(0.975,plx$df)*plx$se, lty=2)
lines(cars$speed,plx$fit + qt(0.975,plx$df)*plx$se, lty=2)

lowess example

edited Mar 28 '14 at 17:07

answered Mar 28 '14 at 16:37

Henry David Thorough

900
8
20

7

Since `predict.loess(...,se=T)` also returns estimated df for the model, it's probably more accurate in general to use: `plx$fit +/- qt(0.975,plx$df)*plx$se` – jlhoward Mar 28 '14 at 17:02
Thanks, but you know what my data is unlike the beautiful 'car' data. Its chemical element concentrations which have lower limits of sometimes 0.10. The drawback of LOESS is that it predicts negative values (which is impossible), but LOWESS doesn't since it is more robust. I tried changing the loess.control parameters, but no output. I found one tricky answer here: http://stackoverflow.com/questions/21166218/setting-an-upper-bound-of-0-on-a-3d-loess-smoothing-with-negative-values-in-r. But adding extra concentrations are unrealistic, it is like contaminating environmental data. – ToNoY Mar 28 '14 at 21:40
2

Well, then you should post a minimal working example demonstrating your particular problem. – Henry David Thorough Mar 29 '14 at 00:18
Is there some way to get the slope of the fitted curve and its uncertainty? – Simon Woodward Mar 15 '21 at 01:43

How to get the confidence intervals for LOWESS fit using R?

1 Answers1

Linked