15

The scikit-learn package provides the functions Lasso() and LassoCV() but no option to fit a logistic function instead of a linear one...How to perform logistic lasso in python?

Fringant
  • 525
  • 1
  • 5
  • 17

3 Answers3

29

The Lasso optimizes a least-square problem with a L1 penalty. By definition you can't optimize a logistic function with the Lasso.

If you want to optimize a logistic function with a L1 penalty, you can use the LogisticRegression estimator with the L1 penalty:

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
log = LogisticRegression(penalty='l1', solver='liblinear')
log.fit(X, y)

Note that only the LIBLINEAR and SAGA (added in v0.19) solvers handle the L1 penalty.

TomDLT
  • 4,346
  • 1
  • 20
  • 26
  • 4
    lasso isn't only used with least square problems. any likelihood penalty (L1 or L2) can be used with any likelihood-formulated model, which includes any generalized linear model modeled with an exponential family likelihood function, which includes logistic regression. – william_grisaitis Dec 19 '19 at 17:09
  • 5
    Agreed. Originally defined for least squares, Lasso regularization is easily extended to a wide variety of statistical models. In scikit-learn though, the `Lasso` class only includes least-square. Other classes include L1 regularization (`LogisticRegression`, `NMF`, ...), but it is called "L1 regularization", and not "Lasso". – TomDLT Jan 23 '20 at 18:17
  • ah ok. i thought you were referring to lasso generally. – william_grisaitis Apr 21 '20 at 14:54
2

You can use glmnet in Python. Glmnet uses warm starts and active-set convergence so it is extremely efficient. Those techniques make glmnet faster than other lasso implementations. You can download it from https://web.stanford.edu/~hastie/glmnet_python/

Kaushalya
  • 77
  • 1
  • 8
Penning Yu
  • 151
  • 1
  • 4
1

1 scikit-learn: sklearn.linear_model.LogisticRegression

sklearn.linear_model.LogisticRegression from scikit-learn is probably the best:

as @TomDLT said, Lasso is for the least squares (regression) case, not logistic (classification).

from sklearn.linear_model import LogisticRegression

model = LogisticRegression(
    penalty='l1',
    solver='saga',  # or 'liblinear'
    C=regularization_strength)

model.fit(x, y)

2 python-glmnet: glmnet.LogitNet

You can also use Civis Analytics' python-glmnet library. This implements the scikit-learn BaseEstimator API:

# source: https://github.com/civisanalytics/python-glmnet#regularized-logistic-regression

from glmnet import LogitNet

m = LogitNet(
    alpha=1,  # 0 <= alpha <= 1, 0 for ridge, 1 for lasso
)
m = m.fit(x, y)

I'm not sure how to adjust the penalty with LogitNet, but I'll let you figure that out.

3 other

PyMC

you can also take a fully bayesian approach. rather than use L1-penalized optimization to find a point estimate for your coefficients, you can approximate the distribution of your coefficients given your data. this gives you the same answer as L1-penalized maximum likelihood estimation if you use a Laplace prior for your coefficients. the Laplace prior induces sparsity.

the PyMC folks have a tutorial here on setting something like that up. good luck.

william_grisaitis
  • 5,170
  • 3
  • 33
  • 40