0

I was doing a benchmark for myself that I encountered this interesting thing. I am trying to get the first 30 keys of a dictionary, and I have written three ways to get it as follows:

import time
dic = {str(i): i for i in range(10 ** 6)}

start_time = time.time()
x = list(dic.keys())[0:30]
print(time.time() - start_time)

start_time = time.time()
y = list(dic.keys())
x = y[0:30]
print(time.time() - start_time)

start_time = time.time()
z = dic.keys()
y = list(z)
x = y[0:30]
print(time.time() - start_time)

The results are:

0.015970945358276367
0.010970354080200195
0.01691460609436035

Surprisingly, the second method is much faster! Any thoughts on this?

Diamond
  • 598
  • 5
  • 15
  • 3
    This is not a good way of benchmark this. Try running them 1000 times and make an average (which is also not a perfect way but at least better). – Netwave May 09 '20 at 11:37
  • 1
    Could be warm-up effects making the very first method look bad; it's the first call to `dic.keys()`. [Idiomatic way of performance evaluation?](https://stackoverflow.com/q/60291987) – Peter Cordes May 09 '20 at 11:47

2 Answers2

3

Using Python's timeit module to measure various alternatives. I added mine which doesn't convert the keys to list:

from timeit import timeit

dic = {str(i): i for i in range(10 ** 6)}

def f1():
    x = list(dic.keys())[0:30]
    return x

def f2():
    y = list(dic.keys())
    x = y[0:30]
    return x

def f3():
    z = dic.keys()
    y = list(z)
    x = y[0:30]
    return x

def f4():
    x = [k for _, k in zip(range(30), dic.keys())]
    return x

t1 = timeit(lambda: f1(), number=10)
t2 = timeit(lambda: f2(), number=10)
t3 = timeit(lambda: f3(), number=10)
t4 = timeit(lambda: f4(), number=10)

print(t1)
print(t2)
print(t3)
print(t4)

Prints:

0.1911074290110264
0.20418328599771485
0.18727918600779958
3.5186996683478355e-05
Andrej Kesely
  • 168,389
  • 15
  • 48
  • 91
1

Maybe this is due to inaccuracies in your measure of time. You can use timeit for doing this kind of things:

import timeit

dic = {str(i): i for i in range(10 ** 6)}

# 27.5125/29.0836/26.8525
timeit.timeit("x = list(dic.keys())[0:30]", number=1000, globals={"dic": dic})

# 28.6648/26.4684/30.9534
timeit.timeit("y = list(dic.keys());x=y[0:30]", number=1000)

# 31.7345/29.5301/30.7541
timeit.timeit("z=dic.keys();y=list(z);x=y[0:30]", number=1000, globals={'dic': dic})

The comments show the times I got when running the same code 3 different times. As you can see, even by performing a large number of repetitions, it is possible to obtain quite large variations in time measured. This can be due to several different things:

  1. An item can be in the cache of your processor or not.
  2. Your processor can be occupied doing several other things.
  3. Etc...

As stated by @Andrej Kesely, your bottleneck is due to the fact that you cast your dictionary keys into a list. By doing so, Python goes through the entire dictionary keys, because that's how it converts something to a list generally. Hence, by avoiding this, you can get much better results.

Tristan Nemoz
  • 1,844
  • 1
  • 6
  • 19