21

I have a regression model with binary outcome. I fitted the model with glmnet and got the selected variables and their coefficients.

Since glmnet doesn't calculate variable importance, I would like to feed the exact output (selected variables and their coefficients) to glm to get the information (Standard errors, etc).

I searched r documents, it seems I can use "method" option in glm to specify user defined function. But I failed to do so, could someone help me with this?

smci
  • 32,567
  • 20
  • 113
  • 146
TongZZZ
  • 756
  • 2
  • 8
  • 20
  • reproducible example please? http://tinyurl.com/reproducible-000 – Ben Bolker Oct 17 '12 at 15:39
  • I don't have an example right now, but, to simplify the question a little bit. Suppose I have a formula output from glmnet as y=2.3*x1+3.1*x2+0.9*x3. How do I specify this formular in glm so that it is manipulated? – TongZZZ Oct 17 '12 at 15:44
  • that's not reproducible, but it gives me a slightly better idea. I think you may not be able to do what you want: that is, you could feed a fully specified model to glm by specifying it as an offset term, but I'm not sure it could compute variable importance from that starting point. Have you thought about using the `caret` package, which does a form of shrinkage by averaging across models *and* provides info on variable importance? – Ben Bolker Oct 17 '12 at 15:49
  • I will check out "caret", and also could you help me with some source on how to use the offset term in glm? – TongZZZ Oct 17 '12 at 15:53
  • 1
    see `?glm`. I really don't think `offset` is going to work for you, though. If you give a **reproducible** example maybe someone else will chime in. – Ben Bolker Oct 17 '12 at 16:17

2 Answers2

41

"It is a very natural question to ask for standard errors of regression coefficients or other estimated quantities. In principle such standard errors can easily be calculated, e.g. using the bootstrap.

Still, this package deliberately does not provide them. The reason for this is that standard errors are not very meaningful for strongly biased estimates such as arise from penalized estimation methods. Penalized estimation is a procedure that reduces the variance of estimators by introducing substantial bias. The bias of each estimator is therefore a major component of its mean squared error, whereas its variance may contribute only a small part.

Unfortunately, in most applications of penalized regression it is impossible to obtain a sufficiently precise estimate of the bias. Any bootstrap-based calculations can only give an assessment of the variance of the estimates. Reliable estimates of the bias are only available if reliable unbiased estimates are available, which is typically not the case in situations in which penalized estimates are used.

Reporting a standard error of a penalized estimate therefore tells only part of the story. It can give a mistaken impression of great precision, completely ignoring the inaccuracy caused by the bias. It is certainly a mistake to make confidence statements that are only based on an assessment of the variance of the estimates, such as bootstrap-based confidence intervals do."

Jelle Goeman, Ph.D. Leiden University, Author of the Penalized package in R.

franciscojavierarceo
  • 476
  • 8
  • 12
1

There is CRAN packages hdi and selectiveInference which provide inference for high-dimensional models, you might want to take a look at those... I've also seen people just run a glm using the predictors selected by glmnet, but this doesn't take into account the uncertainty produced by the selection process of the best model itself...

Tom Wenseleers
  • 7,535
  • 7
  • 63
  • 103