Python: is there a way to import a variable using timeit.timeit()?

Question

Suppose I have some function that takes an array and changes every element to be 0.

def function(array):
    for i in range(0,len(array)):
        array[i] = 0
return array

I want to test how long this function takes to run on a random array, which I wish to generate OUTSIDE of the timeit test. In other words, I don't want to include the time it takes to generate the array into the time.

I first store a random array in a variable x and do:

timeit.timeit("function(x)",setup="from __main__ import function")

But this gives me an error: NameError: global name 'x' is not defined

How can I do this?

Possible duplicate of [Getting "global name 'foo' is not defined" with Python's timeit](https://stackoverflow.com/questions/551797/getting-global-name-foo-is-not-defined-with-pythons-timeit) — sds, Sep 20 '17 at 16:22
For Python 3.5+ check out the answer [here](https://stackoverflow.com/a/48053040/9059420) — Darkonaut, Feb 09 '19 at 22:50

score 23 · Answer 1 · answered Sep 18 '13 at 21:36

23

Import x from __main__ as well:

timeit.timeit("function(x)", setup="from __main__ import function, x")

Just like function, x is a name in the __main__ module, and can be imported into the timeit setup.

answered Sep 18 '13 at 21:36

Martijn Pieters

1,048,767
296
4,058
3,343

wow, that was silly of me not to think of that. Thank you, Martijn! – Sep 18 '13 at 21:39
How would I include multiple lines of setup? – Polydynamical Oct 09 '20 at 19:03
1

@cbracketdash: usually by executing your setup *before you get to that point*. I only ever the `setup` argument to set up the namespace, so import names from `__main__`. – Martijn Pieters Oct 09 '20 at 19:40

abarnert · Answer 2 · 2013-09-18T22:18:03.577

12

You can avoid this problem entirely if you pass timeit a function instead of a string. In that case, the function executes in its normal globals and closure environment. So:

timeit.timeit(lambda: function(x))

Or, if you prefer:

timeit.timeit(partial(function, x))

(See here for details. Note that it requires Python 2.6+, so if you need 2.3-2.5, you can't use this trick.)

As the documentation says, "Note that the timing overhead is a little larger in this case because of the extra function calls."

This means that it makes timeit itself run slower. For example:

>>> def f(): pass
>>> timeit.timeit('timeit.timeit("f()", setup="from __main__ import f")', setup='import timeit', number=1000)
91.66315175301861
>>> timeit.timeit(lambda: timeit.timeit(f), number=100)
94.89793294097762

However, it doesn't affect the actual results:

>>> timeit.timeit(f, number=100000000)
8.81197881908156
>>> timeit.timeit('f()', setup='from __main__ import f', number=100000000)
8.893913001054898

(In the rare cases where it does, that typically means one version or the other wasn't testing the function the way it will be called in your real code, or was testing the wrong closure or similar.)

Note that the actual time taken inside the function here is about 88 seconds, so we've nearly doubled the overhead of timing code… but it's still only added 3% to the total testing time. And the less trivial f is, the smaller this difference will be.

edited Sep 18 '13 at 22:18

answered Sep 18 '13 at 21:38

abarnert

354,177
51
601
671

1

But note that you need to do this consistently across all tests so as not to let the additional cost of stack frame ops influence your comparisons. – Martijn Pieters Sep 18 '13 at 21:49
@MartijnPieters: In normal cases, there are no extra stack frame ops inside the part being timed, only in the cost of timing itself. For example, with an empty function, I get 29.3ns per loop either way… but running `timeit` on the `timeit` itself, it's taking 94ms instead of 91ms to find that 29.3ns. – abarnert Sep 18 '13 at 22:11
(You might want to double-check my test: `timeit.timeit(lambda: timeit.timeit(f), number=1000)` is trivial; `timeit.timeit('timeit.timeit("f()", setup="from __main__ import f")', setup='import timeit', number=1000)` a bit less so…) – abarnert Sep 18 '13 at 22:12
Is the fact that you can pass `timeit.timeit()` a function instead of a string documented somewhere and if so please point me to it? Thanks. – martineau Sep 18 '13 at 22:54
@martineau: Yes, it's documented exactly where I linked in my answer. (The 3.x version is [here](http://docs.python.org/3/library/timeit.html#timeit.Timer), but it's basically the same.) – abarnert Sep 18 '13 at 22:58
What you linked to in your answer is the documentation for the `timeit.Timer` class not the `timeit.timeit` [function](http://docs.python.org/2.7/library/timeit.html#timeit.timeit). Regardless, neither look like they accept an argument that could be the function to be timed -- or am I missing something? – martineau Sep 18 '13 at 23:04
2

@martineau: *Changed in version 2.6: The stmt and setup parameters can now also take objects that are callable without arguments.* Functions are callable objects. `timeit.timeit()` is documented as a shorthand function for `Timer()` instances. – Martijn Pieters Sep 18 '13 at 23:09
@Martijn: Ah, that must be the "trick" abarnert was alluding to -- guess I should try to pay bit more attention to the "Changed in a version < the ones I'm using" notes. Anyway, thanks! – martineau Sep 18 '13 at 23:20
@martineau: The `timeit` function says "Create a Timer instance with the given… and run its `timeit` method", which tells you to look at `Timer` for what the arguments mean, which is why I linked to `Timer` instead of `timeit`. (Also, note that it's only the 2.6 and 2.7 docs that say "Changed in version 2.6"; in [the current docs](http://docs.python.org/3/library/timeit.html#timeit.Timer) it's just described as part of the functionality.) – abarnert Sep 18 '13 at 23:33
In your "doesn't affect the actual result" paragraph, you're not really showing that the function approach doesn't add overhead, since you're timing the function you created solely to wrap the thing you *really* want to time. Compare instead: `timeit.timeit(lambda: None)` and `timeit.timeit("pass")`. – detly Jul 01 '14 at 13:09

score 3 · Answer 3 · answered Sep 18 '13 at 21:36

3

Import x from __main__:

timeit.timeit("function(x)",setup="from __main__ import function, x")

answered Sep 18 '13 at 21:36

Andrew Clark

202,379
35
273
306

Darkonaut · Answer 4 · 2018-11-17T17:30:57.050

1

With Python 3.5, the optional argument globals has been introduced. It makes it possible to specify the namespace in which the timeit statement shall be executed.

So instead of writing:

timeit.timeit("function(x), setup="from __main__ import function, x")

...you can now write:

timeit.timeit("function(x)", globals=globals())

edited Nov 17 '18 at 17:30

answered Jan 01 '18 at 20:33

Darkonaut

20,186
7
54
65

How would I include multiple lines of setup? – Polydynamical Oct 09 '20 at 19:01
1

@cbracketdash It depends on what you need. If your function only needs access to globals you can do the setup in regular code and use `globals` like shown. Additionally or if you can't just use globals you still can use the `setup` parameter like `setup="print('line1'); print('line2')"` where you use the semicolon to separate lines. – Darkonaut Oct 09 '20 at 19:28

score 0 · Answer 5 · answered Feb 09 '17 at 02:20

Alternatively, you can add x to globals. The good thing about it is that it works in a pdb debugging session:

globals()['x'] = x
timeit.timeit(lambda: function(x))

Note that the timing overhead is a little larger in this case because of the extra function calls. [source]

Python: is there a way to import a variable using timeit.timeit()?

5 Answers5