3

I'm trying to calculate the beta for ridge regression from scratch. But when I compare the results when using the glmnet function, they are different. Here's my code:

library(MASS)
Boston=na.omit(Boston)
x=model.matrix(age~.,Boston)[,-1]
y=as.matrix(Boston$age)
lmodel=lm(y~x, data=Boston)

#ridge with glmnet
library(glmnet)
ridge=glmnet(x,y, alpha=0, lambda = 0.1)
beta_ridge=coef(ridge)


#ridge scratch 
x=scale(x)
y=scale(y)
beta_r=solve(t(x)%*%x + 0.1*diag(dim(x)[2]))%*%t(x)%*%y

I standardized x and y, but I still get different betas in comparison to the betas from the glmnet function.

Majk
  • 87
  • 1
  • 9
  • Did you switch the y and x in lmodel=lm(x~y, data= Boston) – Harlan Nelson Feb 17 '18 at 20:10
  • @HarlanNelson oh ok now I see that, but the result is still incorrect – Majk Feb 17 '18 at 20:41
  • 1
    See https://stats.stackexchange.com/questions/129179/why-is-glmnet-ridge-regression-giving-me-a-different-answer-than-manual-calculat – Julius Vainora Feb 17 '18 at 22:09
  • Does your beta_r=solve(t(x)%*%x + 0.1*diag(dim(x)[2]))%*%t(x)%*%y include the intercept? Your beta_ridge does. It looks like your design matrix does not include the intercept. – Harlan Nelson Feb 17 '18 at 22:56
  • @HarlanNelson the beta_r doesn't include the intercept, only beta_ridge. I thought for ridge regression I don't have to use the intercept! – Majk Feb 17 '18 at 23:02
  • Yes, now that I think about it, that is right, the intercept does not change the column space. – Harlan Nelson Feb 17 '18 at 23:18

0 Answers0