Should I assign a range outside of a loop in Python?

Question

I realize that Python isn't the most performant language, but since this seems like it would be easy, I'm wondering whether it's worthwhile to move a range assignment outside of a for loop if I have nested loops. For example:

for i in range(1000):
    for j in range(1000):
        foo()

versus

r = range(1000)
for i in range(1000):
    for j in r:
        foo()

Will the second one run faster, or will the Python interpreter optimize out the repeated function call in the first example? Also, does the answer change if I were to use xrange? (or Python 3 range). Thanks!

Are they the same? Yes. Otherwise no. – Navith Jul 07 '15 at 00:13 — Navith, Jul 07 '15 at 00:13

score 4 · Answer 1 · answered Jul 07 '15 at 00:11

4

Let's see:

@timeit
def a():
    x = 0
    for i in range(10000):
        for j in range(10000):
            x+=1
    return x

@timeit
def b():
    x = 0
    r = range(10000)
    for i in range(10000):
        for j in r:
            x+=1
    return x

a()
b()

'a' ((), {}) 3.30 sec
'b' ((), {}) 2.64 sec

So yes, the second one appears faster.

answered Jul 07 '15 at 00:11

Hrvoje

362
2
6

now try with `xrange` instead of `range` in (a) and it will get even faster, because no lists get allocated. – P.R. Jul 07 '15 at 00:17
When discussing `range` performance I generally assume Python 3, as Python 2's `range`/`xrange` are non-performant almost by default. – TigerhawkT3 Jul 07 '15 at 00:29
I'd like to see the results of replacing `for j in range_object: x+=1` with just `range_object` to eliminate the noise from iterating through the range and incrementing the counter, since all we're really interested in is the creation or reuse of those `range` objects. – TigerhawkT3 Jul 07 '15 at 00:52

hermansc · Accepted Answer · 2015-07-07T10:03:02.290

Extending on @Hrvoje's answer, here are my benchmarks:

import timeit

NUM_ITERATIONS = 5000
NUM_BENCHMARKS = 50

def a():
    x = 0
    for i in range(NUM_ITERATIONS):
        for j in range(NUM_ITERATIONS):
            x+=1
    return x

def b():
    x = 0
    r = range(NUM_ITERATIONS)
    for i in range(NUM_ITERATIONS):
        for j in r:
            x+=1
    return x

def c():
    x = 0
    r1 = range(NUM_ITERATIONS)
    r2 = range(NUM_ITERATIONS)
    for i in r1:
        for j in r2:
            x+=1
    return x

def d():
    x = 0
    for i in xrange(NUM_ITERATIONS):
        for j in xrange(NUM_ITERATIONS):
            x += 1
    return x

def e():
    x = 0
    r = xrange(NUM_ITERATIONS)
    for i in xrange(NUM_ITERATIONS):
        for j in r:
            x += 1
    return x

print "A: %.4f" % (timeit.timeit(a, number=NUM_BENCHMARKS))
print "B: %.4f" % (timeit.timeit(b, number=NUM_BENCHMARKS))
print "C: %.4f" % (timeit.timeit(c, number=NUM_BENCHMARKS))
print "D: %.4f" % (timeit.timeit(d, number=NUM_BENCHMARKS))
print "E: %.4f" % (timeit.timeit(e, number=NUM_BENCHMARKS))

Results:

A: 81.5383
B: 71.2025
C: 69.2143
D: 94.8806
E: 93.1961

Very much as expected. No real difference in using xrange as it's returning a generator. Slight benefit of predefining the range variable.

Selected as the answer because it covered the difference between predefinition and defining in the loop, as well as the difference between range and xrange, as requested. Thanks! — Davis Yoshida, Jul 07 '15 at 01:07
Note that in Python 2 `range` will allocate the memory for the whole list and that could lead to massive memory usage. — Cloves Almeida, Jul 07 '15 at 01:17

NightShadeQueen · Answer 3 · 2015-07-07T00:22:58.160

The results I got were quite inconclusive

In [1]: def f():
   ...:     for i in range(1000):
   ...:             for j in range(1000):
   ...:                     pass
   ...:         

In [2]: def g():
   ...:     r = range(1000)
   ...:     for i in range(1000):
   ...:             for j in r:
   ...:                     pass
   ...:         

In [3]: import itertools

In [4]: def h():
   ...:     for i in itertools.product(range(1000),range(1000)):
   ...:         pass

In [6]: timeit f()
10 loops, best of 3: 23.6 ms per loop

In [7]: timeit g()
10 loops, best of 3: 22.9 ms per loop

In [8]: timeit h()
10 loops, best of 3: 38.3 ms per loop

I suppose the moral of this story is "don't use itertools, that one's slower"

(Python 3, so range is a generator here)

Python 3's `range` isn't a generator, it's a special object with additional logic. It isn't consumed when iterated through, it doesn't need to be iterated through to perform basic operations like `in`, and so on. — TigerhawkT3, Jul 07 '15 at 00:30

Should I assign a range outside of a loop in Python?

3 Answers3