Lasso on sklearn does not converge

Question

When I run something like

import numpy
from sklearn import linear_model
A= #something
b= #something
clf=linear_model.Lasso(alpha=0.015, fit_intercept=False, tol=0.00000000000001,
          max_iter=10000000000000, positive=True)
clf.fit(A,b)

I get the error:

usr/local/lib/python2.7/dist-packages/scikit_learn-0.14.1-py2.7-linux-x86_64.egg/
sklearn/linear_model/coordinate_descent.py:418: UserWarning: Objective did not
converge. You might want to increase the number of iterations
' to increase the number of iterations')

The interesting thing is that A is never rank defficient. (I think)

score 37 · Accepted Answer · edited Feb 02 '18 at 16:32

37

Try increasing tol.

From the documentation:

tol : float, optional

The tolerance for the optimization: if the updates are smaller than tol, the optimization code checks the dual gap for optimality and continues until it is smaller than tol.

The default for tol is 0.0001 on my version of scikit-learn. I assume that your tolerance is so small that the optimization never reaches a lower value.

edited Feb 02 '18 at 16:32

David Bridgeland

525
1
9
16

answered Dec 19 '13 at 12:40

MB-F

22,770
4
61
116

That's exactly that! Thank you – gota Dec 19 '13 at 13:03
well, how can I pass the arguments in a way it went through all alphas and chose the one with just the lowest error disregards its actual value? – 2xMax Feb 25 '19 at 21:19
what else can i do to get rid of the warning message if I already increased tol and max_iter? – PiCubed Sep 15 '20 at 00:01
1

Can someone kindly tell me what is the meaning of `dual gap` here? Thanks! – Max Wong Mar 01 '21 at 15:47
why is the lasso converging at all isn't it just multiplication of matrices even with the penalty? – criticalth Jun 09 '21 at 12:49
This worked for me...another thing that worked was increasing my noise when using make_regression to create the dataset that I applied the LASSO regression on. – Aloha Churchill Dec 08 '21 at 00:27

Gene M · Answer 2 · 2020-01-23T02:29:30.587

The only thing that SOMETIMES helped me to get rid of the warning was increasing the number of iterations significantly (with a significant increase of training time).

Increasing the tolerance always led to the same warnings, but with larger values in them, and not to getting rid of the warnings. Not sure why.

As an important analytical side note, I interpret getting this warning initially when using Lasso regression as a bad sign, regardless of what happens next.
For me it practically always occurred in the situation when the model was over-fitting, meaning that it performed well on the full training set itself, but then poorly during cross-validation and testing.
Regardless of whether I had supressed the warning (there is a way) or had gotten rid of it "naturally" by increasing the number of iterations, I almost always had to go back and simplify the set of features for Lasso to be effective (and in some cases to abandon Lasso altogether in favor of a different model).

What "different model" are you talking about? Just drop the names, I can look them up by myself and see if they might be useful. Thanks! — MrTomRod, Apr 28 '21 at 09:06

score 2 · Answer 3 · answered Mar 19 '21 at 02:30

2

I've got the same problem. Depending on the data variety, using the option normalize=True also help to make the model converge.

answered Mar 19 '21 at 02:30

Josue Freitas

31
1

score 0 · Answer 4 · answered Oct 18 '21 at 18:48

0

change some default in lasso regression:

from sklearn.linear_model import Lasso
Lasso(normalize=True, tol=1e-2)

and another solution is turning off warnings :))

import warnings
warnings.filterwarnings('ignore')

answered Oct 18 '21 at 18:48

moraei

1,443
1
16
15

Lasso on sklearn does not converge

4 Answers4