0

I wrote this lambda for a sigmoid function:

sigmoid = lambda x: 1 / (1 + np.exp(-x))

I use this on arrays to calculate the sigmoid obviously.

Python gives me a warning:

RuntimeWarning: overflow encountered in exp

for this statement. How do I fix it?

The values that are sent to this lambda are floating point numbers.

Anindya Dutta
  • 1,972
  • 2
  • 18
  • 33
  • Sounds like you should just be using [`scipy.special.expit`](https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.special.expit.html). – user2357112 Sep 10 '17 at 03:23
  • I am working on an assignment which only allows me to work with numpy. The outputs are coming alright, just that I do not know how this warning may affect me. – Anindya Dutta Sep 10 '17 at 03:23
  • 1
    your values of `x` are most likely too large for the exponential - remember the `exp(-x)` of a large negative number is truly large i.e. `exp(|x|)` so you will encounter your problem. – Chinny84 Sep 10 '17 at 03:25
  • The warning means `np.exp(-x)` returned infinity, because floating-point numbers don't go high enough to represent the result. That's fine for this computation, but you could [turn off overflow warnings](https://docs.scipy.org/doc/numpy/reference/generated/numpy.seterr.html) or avoid the overflow if you want. (`scipy.special.expit` wouldn't produce this warning.) – user2357112 Sep 10 '17 at 03:27
  • Possible duplicate of [Deal with overflow in exp using numpy](https://stackoverflow.com/questions/9559346/deal-with-overflow-in-exp-using-numpy) – wwii Sep 10 '17 at 03:29
  • 1
    The only issue with a silently treating these `errors` could be an issue in your model (I am assuming that you are using some sort of regression or ML problem) where not seeing large negative values (where there should not be any) would lead to your results being off.. – Chinny84 Sep 10 '17 at 03:29
  • you should check the value of x in your code before it passes through the sigma function.. – jtitusj Sep 10 '17 at 05:47

2 Answers2

1

numpy.clip

np.clip could also work on a x array all at once

import sys   
import numpy as np

xmax = np.log(sys.float_info.max)

sigmoid = lambda x: 1 / (1 + np.exp(np.clip(-x, a_min=None, a_max=xmax)))

sigmoid(-10000)
Out[155]: 5.5626846462681369e-309
f5r5e5d
  • 3,656
  • 3
  • 14
  • 18
1

You can use numpy.seterr

numpy.seterr() always returns a dictionary with the current handling of different numerical errors, that by default are

>>> seterr()
{'divide': 'warn', 'over': 'warn', 'under': 'ignore', 'invalid': 'warn'}
>>>

To change one of these behaviours and saving the current defaults you can do

>>> def_handlers = np.seterr(over='ignore')

then you can do your sigmoid evaluation and at the end you could restore the defaults

>>> np.seterr(**def_handlers);

Consider that the array of sigmoid values may now contain inf entries, your code has to deal with this eventuality.

gboffi
  • 22,939
  • 8
  • 54
  • 85