3

I have some code that uses large integers in its calculation. As I also use numpy it seems that some of the variables are set as type 'numpy.int64' which means they over flow. How can I get round this?

If you run the code below you will see lines under "debugging information". For example

19 1 72.7204246831 524288
19 2 2437717.7229 274877906944
19 3 149857055799.0 144115188075855872
19 4 1.73379003246e+16 0

where the first two columns are n and w and the last column is meant to be 2**(n*w). Clearly 0 is an overflow error.

How can I get round this?

#!/usr/bin/python
from __future__ import division
from scipy.misc import comb
from  scipy.misc import factorial
import math
import numpy as np

N = 20
X = np.arange(2,N)

def k_loop_n(w,n):
    K = np.arange(0, w+1)
    return (comb(w,K)*(comb(w,K)/2**w)**n).sum()

def w_loop(n):
    print "w loop"
    v = [comb(n,w)*k_loop_n(w,n) for w in range(1,n+1)]
    print v
    return sum(v)

#This is meant to be an upper bound for sum (w choose i)^(n+1), i = 0..w 
def sum_upper(i,n):
    return (i+1)*((2**i)*math.sqrt(2/(i*np.pi))*(1-1/(4*i)))**(n+1)

def w_loop_upv2(n):
    print "w loop upper bound"
    print "debugging info"
    print type(n)
    for w in range(1,n+1):
        print n, w, sum_upper(w,n), 2**(w*n)
    v = [comb(n,w)*sum_upper(w,n)/2**(w*n) for w in range(1,n+1)]
    return sum(v)

def upper_k_loop(w,n):
    K = np.arange(0, w+1)
    return (upperbin(w,K)*(upperbin(w,K)/2**w)**(3*float(n)/np.log(n))).sum()


def upper_w_loop(n):
    v = [upperbin(n,w)*k_loop(w,n) for w in range(1,n+1)]
    return sum(v)


print X
Y = [w_loop(n) for n in X]
Yupper = [w_loop_upv2(n)  for n in X]

print Y
print Yupper
marshall
  • 2,443
  • 7
  • 25
  • 45

2 Answers2

3

If you are going to be working with extremely large numbers, you'll want to be able to control the precision. Take a look at the python library mpmath:

Mpmath is a pure-Python library for multiprecision floating-point arithmetic. It provides an extensive set of transcendental functions, unlimited exponent sizes, complex numbers, interval arithmetic, numerical integration and differentiation, root-finding, linear algebra, and much more. Almost any calculation can be performed just as well at 10-digit or 1000-digit precision, and in many cases mpmath implements asymptotically fast algorithms that scale well for extremely high precision work. Mpmath internally uses Python's builtin long integers by default, but automatically switches to GMP/MPIR for much faster high-precision arithmetic if gmpy is installed or if mpmath is imported from within Sage.

Edit: Since it looks like I may have supplied the code above in a previous answer:

https://stackoverflow.com/a/20576452/249341

I want to reiterate that you should be working with the logrithmic representation here, unless you need the exact number for some combinatorial computation. Use a log(x) instead of x will solve the representation problem in this case without needing to go to mpmath.

Community
  • 1
  • 1
Hooked
  • 84,485
  • 43
  • 192
  • 261
2

Use floats if you're going above the 64 bit integer range. Otherwise if you need integer precision but want values outside of the 64 bit range you can use python Decimal objects. See this answer for details on using Decimal in numpy.

Community
  • 1
  • 1
Pyrce
  • 8,296
  • 3
  • 31
  • 46