I implemented a simpe ridge regression code based on the ridge regression formulas. Then I get some beta coefficients and then using the same dataset I tried it with glmnet but I got different results. I used just one lambda just for simplicity to see if the results will be correct. Here is the simple R code:
#Ridge regression
set.seed(110)
n = 5
p = 2
x=matrix(rnorm(n*p), nrow=n, ncol=p)
y=as.matrix(rnorm(n))
int <- rep(1, length(y))
x <- cbind(int, x)
lambda=0.1
lambda.identity=lambda* diag(ncol(x))
beta_ridge=solve(t(x)%*%x+lambda.identity, t(x)%*%y)
beta_ridge
#using glmnet
library(glmnet)
fit=glmnet(x,y,alpha = 0, lambda = 0.1, standardize = FALSE)
coef(fit)
Here are the results:
#without glmnet
[,1]
int 1.2318648
-0.3213297
0.7596814
#with glmnet
s0
(Intercept) 1.3167044
V1 -0.3809595
V2 0.6795089
I didn't standardize the values so that I thought I would get the same output. I tried to work the glmnet code through but it was very complicated.
What could I be doing wrong in my code? I would very much appreciate any advice.
