How to deal with very large numbers in python (19 digits)

Question

In python I need to be able to cycle through a 19 digit number; what is the typical way of doing this? I'm using python 2.7.x; but will use python 3.x if there is a viable solution.

I have a large number; 1000**5 (and even larger 1000**10) for example; I need to cycle through this number list in a for loop. I am aware of the time it will take; but because I cannot find a way to cycle through such a big number I'm at a loss.

I've tried with xrange, range, and itertools.islice in python 2.7 and receive Overflow errors.

Thanks

Need more description. Cycle through how? As a string? For what purpose? Is this a sequence of numbers? How is the number stored to begin with? — Silas Ray, Sep 28 '12 at 19:48
A 19 digit number is no problem for python. What are you trying to do? — Martijn Pieters, Sep 28 '12 at 19:49
``for i in range(12345678901234567890):`` - just don't expect it to run too quickly. — Gareth Latty, Sep 28 '12 at 19:50
@Lattyware that will kill his memory on any < 3 version. `range` creates the complete list to iterate through. `xrange` creates a generator expression instead (in c), so it won't try to populate a list of length 12345678901234567890 before starting the iteration. In Python 3 `range` is an iterator and is safe to use. See http://docs.python.org/release/3.0.1/whatsnew/3.0.html#views-and-iterators-instead-of-lists — Nisan.H, Sep 28 '12 at 20:00
@Lattyware: On Python 2 it is a `OverflowError` (it is larger than `sys.maxsize`) because `range()` tries to create a list; `xrange()` just doesn't work with number that larger than C `long`. It'll take about century at 3e9 cycles per second otherwise. — jfs, Sep 28 '12 at 20:00
@Nisan.H: no. It won't cause MemoryError. It is just an OverflowError. — jfs, Sep 28 '12 at 20:02
@Lattyware It doesn't even do what's assumed to be needed anyway, as it's not going to provide a sequence of all values with 19 or fewer digits given a base 10 representation in the positive direction, let alone in the negative. — Silas Ray, Sep 28 '12 at 20:03
@J.F.Sebastian I didn't say it will _raise_ a `MemoryError`... It will just fill up the memory, then overflow into the paging file, at which point the entire machine will freeze because it will be bottlenecking on the disk I/O. `range(12345678901234567890)` will require roughly 9.1282e+10 GB of memory to store (at 8 bytes per object, assuming they're all python Ints. Longs will take more memory...) — Nisan.H, Sep 28 '12 at 20:29
@J.F.Sebastian: It depends very much on OS and Python build. With a 64-bit build of Python 2.7.3 on OS X 10.6, for example, range(10**14) raises MemoryError, while range(10**13) quickly starts swapping and bogs down the machine. With a 32-bit build, both those examples give an instant OverflowError. — Mark Dickinson, Sep 28 '12 at 20:56
@J.F.Sebastian: Whoops, I should have read more closely: I see you're talking about the specific `12345678901234567890` number---yes, that indeed should give OverflowError on any common platform. — Mark Dickinson, Sep 28 '12 at 20:59
@MarkDickinson: [source](http://hg.python.org/cpython/file/bff269ee7288/Python/bltinmodule.c#l1985) — jfs, Sep 28 '12 at 21:00
@J.F.Sebastian: I'm pretty familiar with that part of the Python source, thanks. What's your point? — Mark Dickinson, Sep 28 '12 at 21:06
It's a fair point that what I posted will only work in 3.x - it was intended more as a joke than an actual suggestion, but it does run on 3.x — Gareth Latty, Sep 28 '12 at 21:25

Lanaru · Accepted Answer · 2012-09-28T20:09:54.333

3

The following is true for python 3.x:

The int type in python does not have a limit. If you want to iterate through 0,1,2...n where n is a 19-digit number, you can just do:

for i in range(n):
    pass #whatever you like

Although that would take a very long time.

edited Sep 28 '12 at 20:09

answered Sep 28 '12 at 20:01

Lanaru

9,421
7
38
64

[`xrange()` has limit](http://stackoverflow.com/questions/1482480/xrange2100-overflowerror-long-int-too-large-to-convert-to-int) – jfs Sep 28 '12 at 20:06
@user973917 It works for me, are you sure you are on 3.x? If so, what error do you get? – Gareth Latty Sep 28 '12 at 21:26

Bitwise · Answer 2 · 2013-02-08T15:58:29.340

2

Not as pretty as for, but just use while() and you will overcome the problem (plus it will work with whatever version of Python):

i=0
while (i<1000**10):
    i += 1
    # do some stuff

edited Feb 08 '13 at 15:58

answered Sep 28 '12 at 20:29

Bitwise

7,577
6
33
50

score 0 · Answer 3 · answered Sep 28 '12 at 20:07

0

In case you were trying to iterate the single digits:

long_int_string = '34234345456575657'
for digit in [int(x) for x in long_int_string]:
    # do something useful with digit        
    print digit

answered Sep 28 '12 at 20:07

Michael

7,316
1
37
63

score 0 · Answer 4 · answered Sep 28 '12 at 20:08

It sounds like you want to iterate through the individual digits of an integer. For situations like these, it is useful to know that x % 10 is equal to the rightmost digit of a positive integer x in base ten. With that in mind, you can get every digit by iteratively using modulus and reducing the number.

def splitIntoDigits(x):
    digits = []
    while x != 0:
        digits.insert(0, int(x%10))
        x /= 10
    return digits

print splitIntoDigits(15016023042)
[1, 5, 0, 1, 6, 0, 2, 3, 0, 4, 2]

Also, comedy one-line answer:

>>> [int(num/(10**x)%10) for x in range(int(math.log(num,10)),-1,-1)]
[1, 5, 0, 1, 6, 0, 2, 3, 0, 4, 2]

How to deal with very large numbers in python (19 digits)

4 Answers4