Performance of loops, list comprehensions and maps in Python 3

Question

How is it possible to properly compare performance of for loops, list comprehensions and maps in Python 3.6?

In the code below, plain old for loops perform quite well (I used list() to get values from generators). Am I doing something wrong here? The results are in stark contrast to the discussion on Python 2.

import timeit

code_for = """
for i in range(1000):
    hex(i)
"""

code_map = """
list(map(hex, range(1000)))
"""

code_map_lambda = """
list(map(lambda x: hex(x), range(1000)))
"""

code_list_comprehension = """
[hex(x) for x in range(1000)]
"""

print(timeit.timeit(code_for, number=10000))
# 1.1155821208376437

print(timeit.timeit(code_map, number=10000))
# 0.8820606248918921

print(timeit.timeit(code_map_lambda, number=10000))
# 1.7510833400301635

print(timeit.timeit(code_list_comprehension, number=10000))
# 1.1798800542019308

UPDATE: Adding elements to a list in code_for

code_for_2 = """
a = [0] * 1000
for i in range(1000):
    a[i] = hex(i)
"""
# 1.243549756007269

code_for_3 = """
a = []
for i in range(1000):
    a.append(hex(i))
"""
# 1.5462996119167656

jpp · Accepted Answer · 2018-06-25T11:42:44.410

3

A few pointers:

Wrap your code in functions for clarity.
You are missing list creation and appends in code_for. This is the bulk of the cost of using an explicit for loop.
You can then use timeit, or if you have Jupyter notebook, the magic %timeit command.

As below, map without lambda performs best, which makes sense since hex is a built-in. See Python List Comprehension Vs. Map for more details.

def code_for(n):
    res = []
    for i in range(n):
        res.append(hex(i))
    return res

def code_map(n):
    return list(map(hex, range(n)))

def code_map_lambda(n):
    return list(map(lambda x: hex(x), range(n)))

def code_list_comprehension(n):
    return [hex(x) for x in range(n)]

%timeit code_for(10000)                 # 3.19 ms per loop
%timeit code_map(10000)                 # 1.69 ms per loop
%timeit code_map_lambda(10000)          # 3.06 ms per loop
%timeit code_list_comprehension(10000)  # 2.27 ms per loop

edited Jun 25 '18 at 11:42

answered Jun 25 '18 at 11:36

jpp

159,742
34
281
339

Thank you for the reply! Could you please clarify why `list(map(...))` is equivalent to `append()` rather than to setting values in a preallocated list? – Konstantin Jun 25 '18 at 11:39
They are certainly not *equivalent* (hence the performance difference). But the results are identical because they both output lists. If you don't use `list.append`, `code_for` will return nothing, it'll apply `hex` to each item but the results will be forever lost. You should look at [this post](https://stackoverflow.com/questions/1247486/python-list-comprehension-vs-map) for general details on these performance differences. – jpp Jun 25 '18 at 11:42
I'm asking, because with a preallocated list the results of `code_for` are the same as without putting elements to a list at all. `append()` is slower, of course. – Konstantin Jun 25 '18 at 11:46
@Konstantin, Sorry, I don't understand. Why assume lists are pre-allocated with any of these methods? – jpp Jun 25 '18 at 11:46
Please refer to `code_for_2` and `code_for_3`. But it seems that it's really hard to say what happens under the hood in `list(map(...))`. – Konstantin Jun 25 '18 at 11:52
`map` functions differently. You should have a look at the answers available on similar problems: [**1**](https://stackoverflow.com/a/30245465/9209546), [**2**](https://stackoverflow.com/a/22108640/9209546), [**3**](https://stackoverflow.com/a/31349900/9209546), [**4**](https://stackoverflow.com/a/11616687/9209546). – jpp Jun 25 '18 at 11:57

Performance of loops, list comprehensions and maps in Python 3

1 Answers1