3

I would like to compute 1/(1+exp(x)) for (possibly large) x. This is a well behaved function between 0 and 1. I could just do

import numpy as np
1.0/(1.0+np.exp(x))

but in this naive implementation np.exp(x) will likely just return 0 or infinity for large x, depending on the sign. Are there functions available in python that will help me out here?

I am considering implementing a series expansion and series acceleration, but I am wondering if this problem has already been solved.

ali_m
  • 71,714
  • 23
  • 223
  • 298
benbo
  • 1,471
  • 1
  • 16
  • 29
  • 3
    Can you give examples of inputs for which you get poor outputs, alongside the outputs that you'd like to get for those inputs? The expression you give is actually fairly well-behaved numerically for almost all `x`. (The only difficult area I can think of is where `np.exp(x)` only just overflows a double, where you'll end up getting zero instead of the correct subnormal value.) – Mark Dickinson Nov 25 '15 at 16:33

2 Answers2

6

You can use scipy.special.expit(-x). It will avoid the overflow warnings generated by 1.0/(1.0 + exp(x)).

Warren Weckesser
  • 110,654
  • 19
  • 194
  • 214
2

Fundamentally you are limited by floating point precision. For example, if you are using 64 bit floats:

fmax_64 = np.finfo(np.float64).max  # the largest representable 64 bit float
print(np.log(fmax_64))
# 709.782712893

If x is larger than about 709 then you simply won't be able to represent np.exp(x) (or 1. / (1 + np.exp(x))) using a 64 bit float.

You could use an extended precision float (i.e. np.longdouble):

fmax_long = np.finfo(np.longdouble).max
print(np.log(fmax_long))
# 11356.5234063

The precision of np.longdouble may vary depending on your platform - on x86 it is usually 80 bit, which would allow you to work with x values up to about 11356:

func = lambda x: 1. / (1. + np.exp(np.longdouble(x)))
print(func(11356))
# 1.41861159972e-4932

Beyond that you would need to rethink how you're computing your expansion, or else use something like mpmath which supports arbitrary precision arithmetic. However this usually comes at the cost of much worse runtime performance compared with numpy, since vectorization is no longer possible.

ali_m
  • 71,714
  • 23
  • 223
  • 298
  • Thanks, I'll take a look at mpmath. I'll have to figure out what kind of precision will really be necessary for my task. – benbo Nov 25 '15 at 16:27
  • 1
    You might get a more helpful answer if you asked another question explaining the full problem you are trying to solve - maybe there is a way to avoid computing `exp(x)` altogether? – ali_m Nov 25 '15 at 16:29