Python 3: Loops, list comprehension and map slower compared to Python 2?

Question

I am currently learning Python 3 and thought some speed comparison could be neat. Thus, I created some in-place and some temporary list functions, which simply add 1 to a long list of integers.

However, the results were really surprising to me...it seems that Python 3 is slower in every use case: different tested for loops, a while loop, the map function, and list comprehensions.

Of course (see the code below) it just comparison of in-place list mutations or loops that build up and return a temporary list...so e.g. the "standard" map of Python 3, that just creates an iterator, is still fast, but at some point one would always like to print, write or somehow visualize the data. Hence, something like list(map()) should be relatively common in Python 3?!

What am I missing here?

Is Python 3 really slower than Python 2 for loops, comprehensions and map? If so, why?

Python 3.4 timings:

> python3.4 func_speed_tests.py
list_comprehension.........1.2785s
for_loop...................1.9988s
for_loop_append............1.8471s
for_loop_range.............1.7585s
while_loop.................2.2539s
calc_map...................2.0763s

Python 2.7 timings:

> python2.7 func_speed_tests.py
list_comprehension.........0.9472s
for_loop...................1.2821s
for_loop_append............1.5663s
for_loop_range.............1.3101s
while_loop.................2.1914s
calc_map...................1.9101s

Here is the code:

import timeit
from random import shuffle

# generate a long list of integers
mlst = list(range(0, 1000000))

# randomly sort the list
shuffle(mlst)

# number of benchmark passes
n = 10


#####################
# functions to time #
#####################

def list_comprehension(lst):
    return [x + 1 for x in lst]


def for_loop(lst):
    for k, v in enumerate(lst):
        lst[k] = v + 1


def for_loop_append(lst):
    tmp = []
    for item in lst:
        tmp.append(item + 1)


def for_loop_range(lst):
    for k in range(len(lst)):
        lst[k] += 1
    return lst


def while_loop(lst):
    it = iter(lst)
    tmp = []
    while it:
        try:
            tmp.append(next(it)+1)
        except StopIteration:
            break
    return tmp


def calc_map(lst):
    lst = map(lambda x: x + 1, lst)
    return list(lst)


############################################
# helper functions for timing and printing #
############################################   

def print_time(func_name):
    print("{:.<25}{:.>8.4f}s".format(func_name, timer(func_name)))


def timer(func):
    return timeit.timeit(stmt="{}(mlst)".format(func),
                         setup="from __main__ import mlst, {}".format(func),
                         number=n)

#################
# run the tests #
#################

print_time("list_comprehension")
print_time("for_loop")
print_time("for_loop_append")
print_time("for_loop_range")
print_time("while_loop")
print_time("calc_map")

are you using `range()` for both python 2 and 3? or `xrange()` for python2 and `range()` for python3? — Lafexlos, Mar 28 '16 at 11:49
Yes, I am running the exact same script. That's why I am using `list(map())` for example, so that it is compatible with Python 2 and 3, and does the same thing in both languages. — daniel451, Mar 28 '16 at 11:50
In fact, since `range()` generates a list rather than an iterator in Python 2, I thought at least the loops using `range()` should be slower in Python 2...but everything is faster o.O — daniel451, Mar 28 '16 at 11:52
@ascenator The creation of the list using `range()` in Python 2 is slower but once the list is there, iterating over the list is faster since it’s internally just a sequencial memory access. The generator solution has an overhead for the iterator protocol, making it likely a bit slower. — poke, Mar 28 '16 at 11:54
@poke ah ok, so the *space complexity* for `range()` in Python 2 is higher, because of the full list in the memory but the *time complexity* is lower, because the list iteration is faster than the iterator?! Hence, Python 2 is faster but uses more memory in those cases. — daniel451, Mar 28 '16 at 11:58
Related: http://stackoverflow.com/questions/31548680/why-is-python-3-is-considerably-slower-than-python-2 – but in general, there is no clear indication: Some things are slower due to how Python 3 works (e.g. `long` being the new `int`, `unicode` being the new `str`), but there are also parts that are faster. It’s difficult to compare, and I don’t think small functional benchmarks will ultimately help you decide. — poke, Mar 28 '16 at 12:05

score 1 · Answer 1 · answered Mar 28 '16 at 11:52

1

Python3 is slightly slower that Python2. Check with pystone benchmark. If speed is of concern stick with Python 2.7.0 Release which has better third party package support.

answered Mar 28 '16 at 11:52

Sandeep

28,307
3
32
24

Ok, thanks. At least we have something to go on with this. Do you have any additional sources or ideas yourself *why* this is happening? – daniel451 Mar 28 '16 at 12:00

Python 3: Loops, list comprehension and map slower compared to Python 2?

1 Answers1

Linked