How to use python timeit when passing variables to functions?

Question

I'm struggling with this using timeit and was wondering if anyone had any tips

Basically I have a function(that I pass a value to) that I want to test the speed of and created this:

if __name__=='__main__':
    from timeit import Timer
    t = Timer(superMegaIntenseFunction(10))
    print t.timeit(number=1)

but when I run it, I get weird errors like coming from the timeit module.:

ValueError: stmt is neither a string nor callable

If I run the function on its own, it works fine. Its when I wrap it in the time it module, I get the errors(I have tried using double quotes and without..sameoutput).

any suggestions would be awesome!

Thanks!

score 138 · Accepted Answer · edited Jan 18 '19 at 08:13

138

Make it a callable:

if __name__=='__main__':
    from timeit import Timer
    t = Timer(lambda: superMegaIntenseFunction(10))
    print(t.timeit(number=1))

Should work

edited Jan 18 '19 at 08:13

Martin Thoma

124,992
159
614
958

answered Sep 23 '11 at 03:01

Pablo

8,644
2
39
29

That worked! Thanks so much. I need to figure out what lambda does..seems like that made the difference. Thanks Pablo – Lostsoul Sep 23 '11 at 03:03
7

if only this were in the documentation somewhere – endolith Oct 03 '13 at 23:58
21

Oh but lambda adds some overhead, so not ideal for testing small things. `timeit 5*5` is 33 ns while `timeit (lambda: 5*5)()` is 233 ns. – endolith Oct 04 '13 at 14:50
This is perfect. I had a problem just calling `timeit` with the lambda (the script froze; the method I was measuring the execution time of is the `retrbinary` for downloading through FTP using the `ftplib`). Once I used the `Timer` object it suddenly worked like a charm. No idea what happened... :D – rbaleksandar Jul 20 '18 at 13:10

score 24 · Answer 2 · answered Sep 23 '11 at 03:29

Timer(superMegaIntenseFunction(10)) means "call superMegaIntenseFunction(10), then pass the result to Timer". That's clearly not what you want. Timer expects either a callable (just as it sounds: something that can be called, such as a function), or a string (so that it can interpret the contents of the string as Python code). Timer works by calling the callable-thing repeatedly and seeing how much time is taken.

Timer(superMegaIntenseFunction) would pass the type check, because superMegaIntenseFunction is callable. However, Timer wouldn't know what values to pass to superMegaIntenseFunction.

The simple way around this, of course, is to use a string with the code. We need to pass a 'setup' argument to the code, because the string is "interpreted as code" in a fresh context - it doesn't have access to the same globals, so you need to run another bit of code to make the definition available - see @oxtopus's answer.

With lambda (as in @Pablo's answer), we can bind the parameter 10 to a call to superMegaIntenseFunction. All that we're doing is creating another function, that takes no arguments, and calls superMegaIntenseFunction with 10. It's just as if you'd used def to create another function like that, except that the new function doesn't get a name (because it doesn't need one).

score 20 · Answer 3 · answered Sep 23 '11 at 02:57

20

You should be passing a string. i.e.

t = Timer('superMegaIntenseFunction(10)','from __main__ import superMegaIntenseFunction')

answered Sep 23 '11 at 02:57

Austin Marshall

2,991
16
14

Thanks for the answer oxtopus! it doesn't work when I wrap it in quotes so its a string I get this error: NameError: global name 'superMegaIntenseFunction' is not defined. What else do you think I can try? – Lostsoul Sep 23 '11 at 02:59
Corrected the answer to include the setup arg. (http://docs.python.org/library/timeit.html#timeit.Timer) – Austin Marshall Sep 23 '11 at 03:03

score 3 · Answer 4 · answered Nov 30 '18 at 21:38

One way to do it would be by using partial so that the function, 'superMegaIntenseFunction' is used as a callable (ie without the ()) in the timer or directly inside timeit.timeit. Using partial will pass the argument to the function when it will be call by the timer.

from functools import partial
from timeit import timeit

print(timeit(partial(superMegaIntenseFunction, 10), number=1))

score 1 · Answer 5 · answered Feb 09 '17 at 02:29

A note for future visitors. If you need to make it work in pdb debugger, and superMegaIntenseFunction is not in the global scope, you can make it work by adding to globals:

globals()['superMegaIntenseFunction'] = superMegaIntenseFunction
timeit.timeit(lambda: superMegaIntenseFunction(x))

Note that the timing overhead is a little larger in this case because of the extra function calls. [source]

How to use python timeit when passing variables to functions?

5 Answers5

Linked