Issues with using timeit in ipython

Question

I was quickly trying to time 2 functions in ipython, m1() and m2() doing the same task with 2 different implementation.

In [23]: %timeit for x in range(100): m1(a)
10000 loops, best of 3: 57.6 us per loop

In [24]: %timeit for x in range(100): m2(a)
10000 loops, best of 3: 108 us per loop

Result: the first implementation is almost 2x faster. So far, so good.

Out of curiousity, I changed the range of the for loop above, and now I am at a loss making sense of the output.

In [25]: %timeit for x in range(1): m2(a)
1000000 loops, best of 3: 1.29 us per loop

In [26]: %timeit for x in range(10): m2(a)
100000 loops, best of 3: 10.8 us per loop

In [27]: %timeit for x in range(1000): m2(a)
1000 loops, best of 3: 1.06 ms per loop

What exactly is the for loop doing here? And why do the value of the number of loops decrease on increasing the range value?

PS: I was using this as the reference. Also, please edit the title to something better if it doesn't exactly convey my question.

Why are you using the `for` loop here? It is totally enough to do `%timeit m1(a)`! Otherwise you time the for loop as well. — Jakob, Oct 20 '13 at 06:38
Yup, I didn't realise initially what the for loop was doing :| — Anshul Goyal, Oct 20 '13 at 06:39

score 2 · Accepted Answer · answered Oct 20 '13 at 04:04

2

timeit is counting the execution time for the entire block.

So what you are seeing is:

running m2(a) 1 time takes 1.29 us
running m2(a) 10 times takes 10.8 us
running m2(a) 1000 times takes 1.06 ms

Which makes some sense, since 1.06ms = 1060 us, roughly 1000x the baseline (and 10.8 us is roughly 10x the baseline)

As for the number of loops, timeit aims to run within a reasonable time:

$ python -mtimeit -h
...
If -n is not given, a suitable number of loops is calculated by trying
successive powers of 10 until the total time is at least 0.2 seconds.

answered Oct 20 '13 at 04:04

SheetJS

22,470
12
65
75

That explains the output, but doesn't answer either of my questions - `What exactly is the for loop doing here?` and `why do the value of the number of loops decrease on increasing the range value?` – Anshul Goyal Oct 20 '13 at 04:05
@ansh0l updated my response -- it's explained in the help for `timeit` – SheetJS Oct 20 '13 at 04:08
1

Thanks, that explains it. I wrote an answer for detailed explanation for my future self :) – Anshul Goyal Oct 20 '13 at 06:21

score 1 · Answer 2 · answered Oct 20 '13 at 06:20

So I finally figured it out what is happening, thanks to @Nirk's answer.

In [26]: %timeit for x in range(100): m2(a)
10000 loops, best of 3: 108 us per loop

Here,

%timeit => ipython magic call

for x in range(100): m2(a) => the statement being executed. Based on the range value, time for execution increases/decreases for each run

10000 loops => Minimum number of loops that timeit will implicilty run, based on timeit modules constraint of minimum 0.2 s of total time

best of 3: 108 us per loop => average time taken by the best 3 loop run by timeit.

Assuming time for each run is same as average of best 3, time for each run = 108 us

minimum loops needed = 10^x, where x is minimum positive integer satisfying 1.08 * (10^-4) * (10^x) > 2 * (10^-1)

i.e, the minimum x for which (10^x) > 1.85 * (10^3) => x = 4

Hence minimum loops needed = 10^x = 10000 loops.

Issues with using timeit in ipython

2 Answers2

Linked