1

I am using Logistic regression for Mnist digit classification and am using statsmodel.api library to fit the parameters but the Logit.fit() still throws an overflow warning.Below is the error I am getting on Windows10,python 2.7 using library downloaded from http://www.lfd.uci.edu/~gohlke/pythonlibs/.

C:\Python27\lib\site-packages\statsmodels\discrete\discrete_model.py:1213: RuntimeWarning: overflow encountered in exp   return 1/(1+np.exp(-X)) C:\Python27\lib\site-packages\statsmodels\discrete\discrete_model.py:1263: RuntimeWarning: divide by zero encountered in log   return np.sum(np.log(self.cdf(q*np.dot(X,params)))) Warning: Maximum number of iterations has been exceeded.
         Current function value: inf
         Iterations: 35 Traceback (most recent call last):   File "code.py", line 44, in <module>
    result1 = logit1.fit()   File "C:\Python27\lib\site-packages\statsmodels\discrete\discrete_model.py", line 1376, in fit
    disp=disp, callback=callback, **kwargs)   File "C:\Python27\lib\site-packages\statsmodels\discrete\discrete_model.py", line 203, in fit
    disp=disp, callback=callback, **kwargs)   File "C:\Python27\lib\site-packages\statsmodels\base\model.py", line 434, in fit
    Hinv = np.linalg.inv(-retvals['Hessian']) / nobs   File "C:\Python27\lib\site-packages\numpy\linalg\linalg.py", line 526, in inv
    ainv = _umath_linalg.inv(a, signature=signature, extobj=extobj)   File "C:\Python27\lib\site-packages\numpy\linalg\linalg.py", line 90, in _raise_linalgerror_singular
    raise LinAlgError("Singular matrix") numpy.linalg.linalg.LinAlgError: Singular matrix
Amir
  • 10,600
  • 9
  • 48
  • 75
ayush gupta
  • 607
  • 1
  • 6
  • 14
  • Please, provide your code and data. (See http://stackoverflow.com/help/mcve ) – Ilya V. Schurov Oct 17 '16 at 07:54
  • The problem is not an overflow, but a "singular matrix", please see this answer http://stackoverflow.com/questions/17105154/calculate-logistic-regression-in-python – borowis Oct 17 '16 at 07:54

1 Answers1

0

It is because the input matrix is a singular matrix during the logistic modelling computation, hence, you have the LinAlgError. If you apply some regularisation to that input, it will works fine. An example is to use 'l1' as in Stat models, or 'l2' as the default option in Logistic Regression model of sklearn.

Maneesha Indrachapa
  • 882
  • 5
  • 18
philipvn
  • 5
  • 5