What is the best way to get the execution time spent of a single line of code?

Question

I am trying to get the execution time spent on a single line of code. Doing this for multiple lines and/or writing code every time like in the example below is just frustrating. Is there a better way to do this? See the example below.

import pandas as pd
import datetime

start_time = datetime.datetime.now()
df = pd.read_csv('dummy.csv')
print('Time spent reading: {}'.format(datetime.datetime.now() - start_time))
start_time = datetime.datetime.now()
df.head(n=100)
print('Time spent printing top 100: {}'.format(datetime.datetime.now() - start_time))

Use the [profiler](https://docs.python.org/3.7/library/profile.html) — Ocaso Protal, Aug 28 '19 at 11:03
What is your goal here, to improve program performance, or just to give the user of your program some information on how long different tasks took? — Martijn Pieters, Aug 28 '19 at 11:05
The goal is to get how long it take to execute single line of code without writing the same pieces of code again and again. Like: start_time = datetime.datetime.now() — MehmedB, Aug 28 '19 at 11:06
@MehmedB: that *doesn't tell us why you want to do that*. There are different reasons why you want to measure the time a line of code takes. Those different reasons lead to different answers. — Martijn Pieters, Aug 28 '19 at 13:28

Thomas · Accepted Answer · 2019-08-28T12:55:57.723

Python has variadic arguments (as well as keyword arguments), argument defaults, and has first-class functions. Basically this means you can pass in a function and optionally all of its arguments and keyword arguments into another function.

from datetime import datetime

def record_time(task, args=(), kwargs={}):
    start_time = datetime.now()
    task(*args, **kwargs)

    return datetime.now() - start_time

print(record_time(print, ("test", )))

Just remember that a 1-ary tuple needs to be written as ('foo', ) and not as ('foo').

You can also use context managers that will allow you to time blocks of code rather than just individual functions.

import contextlib

from datetime import datetime
from time import sleep

@contextlib.contextmanager
def record_time():
    try:
        start_time = datetime.now()
        yield
    finally:
        print(datetime.now() - start_time)

with record_time():
    # Do a lot of stuff here...
    sleep(2)
    # Do more stuff if you like.

with record_time(): sleep(2)

By nature, it is impossible to actually return the time this way but printing it isn't hard.

I totally forgot context managers! Thank you so much for a working example! — MehmedB, Aug 28 '19 at 20:04

score 1 · Answer 2 · answered Aug 28 '19 at 11:50

1

You could use the module timeit

Example use

import timeit

timeit.timeit('"-".join(str(n) for n in range(100))', number=10000)

Out[1]: 0.25778120000006766

Thos code worked fine when using python 3.

You can run multiple cycles to be sure of any variability, and of course you can substitute the code between the ' ' with your own oneliner then.

More information can be found on the doc page for timeit

answered Aug 28 '19 at 11:50

Nice, I learn about more tools in the Python standard library every day. I tested this, worried that eval would make the code in test less performant, but it was negligible from using the `datetime` subtraction method. – Thomas Aug 28 '19 at 12:48
1

Note that running `timeit()` on a function that depends heavily on outside processes like I/O, such `read_csv` in the question, is not going to get you very accurate timings. – Martijn Pieters Aug 28 '19 at 13:29
I was aware of timeit but I was looking for a way to 'timeit' a single line in a more complex algorithm where timing a single line for many times is almost unpractical due to time it take to execute it just for once. There are a lot of steps to the line of interest. – MehmedB Aug 28 '19 at 20:07

What is the best way to get the execution time spent of a single line of code?

2 Answers2