Overflow Error: ..long int too large

Question

import math
a = 100
b = 110
e = 2.71828
x = (e**-a)*(a**b)/math.factorial(b)
print round(x, 5)

when a and b are large I get this message:

Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
OverflowError: long int too large to convert to float

`math.factorial(b)` == 198! == 1.98..e370 that's bigger than Double.MaxValue — Dmitry Bychenko, Sep 11 '14 at 08:21
A better question might to ask why you need the factorial of 198! — Fredrik Pihl, Sep 11 '14 at 08:29
this is Poisson distribution ,and i need that for numbers between 140 - 220 — Nenad Savic, Sep 11 '14 at 08:44

JulienD · Answer 1 · 2017-02-26T09:00:57.353

4

You seem to be trying to implement a Poisson distribution. For large values of its mean, it is very well approximated by a Gaussian distribution, for which you do not need to compute the factorial (which is way too big as others already said).

http://en.wikipedia.org/wiki/Poisson_distribution#Related_distributions

Edit: the number of events k is independant of the mean, but usually one does not want a probability such as P(200|L) for small L, or one rounds it to zero.

Also have a look at Scipy's implementation, which seems to use the logarithm, another way or avoiding very large numbers: https://github.com/scipy/scipy/blob/v0.14.0/scipy/stats/_discrete_distns.py

Edit: since it was requested, a demo in R (because it is quicker for me in my current setup, but the maths are the same). I took lambda = mean = sigma^2 = 500 (without the continuity correction for n<1000):

pois = rpois(1000, 500)
norm = rnorm(1000, 500, sqrt(500))
plot(density(pois))
lines(density(norm))

edited Feb 26 '17 at 09:00

answered Sep 11 '14 at 08:29

JulienD

7,102
9
50
84

How did you figure out the willings of the OP ? – 0x90 Sep 11 '14 at 08:33
The formula is easy to recognize. I had to approximate Binomial with Poisson for similar reasons. At some point even with Decimal you will reach the limits of your hardware. – JulienD Sep 11 '14 at 08:38
yes this is Poisson, I dont know he was similar with Gaussian , thank you – Nenad Savic Sep 11 '14 at 08:47
It's old, but can you please add the code that demonstrates your suggestion? – 0x90 Feb 26 '17 at 04:53
1

@0x90 Maths don't become too old, and you will find the demonstration in math books. But I added a graphical example using simple R code. – JulienD Feb 26 '17 at 09:02

score 1 · Answer 2 · edited May 23 '17 at 12:25

You are exceeding the max float value (1.7976931348623157e+308).

Use the decimal module:

import math
from decimal import Decimal   
a = 200
b = 198
e = 2.71828
x = Decimal(e**-a)*Decimal(a**b)/math.factorial(Decimal(b))
print round(x, 5)

Output:

0.02806

Compare it with the output that WloframAlpha calculated 0.0280567

In case you want to see the float info, see this:

>>> import sys
>>> print  sys.float_info
sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1)

Interesting. This works for 198 but will fill you whole memory quickly as the number increases, or am I wrong? — JulienD, Sep 11 '14 at 08:40

score 1 · Accepted Answer · answered Sep 11 '14 at 09:43

1

as long as (a**b) >> math.factorial(b) you could just do

(a**b)/math.factorial(b)*(e**-a)

this way the number gets smaller before being converted to float

answered Sep 11 '14 at 09:43

Hugo Walter

3,048
1
16
10

Overflow Error: ..long int too large

3 Answers3