python for loop range(bigint)

Question

In Python, is there some short way to do something like

"for i in range(n)"

when n is too big for Python to actually create the array range(n)?

(short because otherwise I'd just use a while loop)

score 5 · Accepted Answer · answered Jul 01 '10 at 06:15

You could use xrange()... although that is restricted to "short" integers in CPython:

CPython implementation detail: xrange() is intended to be simple and fast. Implementations may impose restrictions to achieve this. The C implementation of Python restricts all arguments to native C longs (“short” Python integers), and also requires that the number of elements fit in a native C long. If a larger range is needed, an alternate version can be crafted using the itertools module: takewhile(lambda x: x<stop, (start+i*step for i in count())).

I don't know whether that restriction also applies to other implementations (or which ones) - but there's a workaround listed...

I know you mention bigint in your question title, but the question body talks about the number being too big to create the array - I suspect there are plenty of numbers which are small enough for xrange to work, but big enough to cause you memory headaches with range.

...or use Python 3, where `range` behaves like `xrange` from Python 2 without the restriction to "short" integers... — Tim Pietzcker, Jul 01 '10 at 06:17
As I just learned from http://stackoverflow.com/questions/94935/what-is-the-difference-between-range-and-xrange there is a limitation of xrange to C longs, so you cannot have upper boundaries greater than 2 ** 31 - 1. — Johannes Charra, Jul 01 '10 at 06:20
@jellybean: Or 2**63-1 on a 64-bit (non-Windows) machine. :) — Mark Dickinson, Jul 01 '10 at 08:43

score 3 · Answer 2 · answered Jul 01 '10 at 06:13

3

I would use a generator function: example forthcoming.

def gen():
    i = 0
    while 1: # or your special terminating logic
        yield i
        i = i + 1


for j in gen():
    do stuff

answered Jul 01 '10 at 06:13

John Weldon

39,849
11
94
127

Also, @Jon Skeet's suggestion to use xrange is a good choice. – John Weldon Jul 01 '10 at 06:17
Not really necessary to reinvent the wheel - or so I thought until I read about xrange's restrictions. So if you're using Python 2, this is a good choice. – Tim Pietzcker Jul 01 '10 at 06:18
Agreed, @Tim. :) Unless you want to exceed the limits of xrange. – John Weldon Jul 01 '10 at 06:21

score 1 · Answer 3 · answered Jul 01 '10 at 08:40

You could upgrade to python3. There, range isn't limited to 'short' integers.

Another workaround would be to use xrange for small integers and add them to some constant inside the loop, e.g.

offset, upperlimit = 2**65, 2**65+100
for i in xrange(upperlimit-offset):
    j = i + offset
    # ... do something with j

score 0 · Answer 4 · answered Jul 01 '10 at 06:18

0

you should always use xrange rather than range for just looping n times over sth., but keep in mind that xrange has also a limit (if it is too small you need to do your own while loop with a counter)

EDIT: too late...

answered Jul 01 '10 at 06:18

xyz-123

2,227
4
21
27

python for loop range(bigint)

4 Answers4