13

I just completed writing my first program in python, i have written all my functions in a single module, i just executed it from command line by giving the input files as argument and it worked. But when i gave a big dataset, my program continuously running for some time. Now my next step is to find which function is taking more time in my module. I could get the time taken by whole program, but i need for each function separately.

I tried to understand timeit and profile modules in python, but as per my understanding, they were giving the time taken by a particular function. Is there a way to know the time taken by each function in my module as a statistics( all at once )?

Thanks in advance.

ds_user
  • 2,139
  • 4
  • 36
  • 71

2 Answers2

21

At the terminal, run

python -m profile -s time file.py

or

python -m cProfile -s time file.py

The second can be faster, and is never worse.

This will give something like:

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       39    0.132    0.003    0.139    0.004 :0(load_dynamic)
      239    0.097    0.000    0.097    0.000 :0(loads)
    541/1    0.083    0.000    3.556    3.556 :0(exec)
       30    0.082    0.003    0.082    0.003 :0(statusBar)
                        ... etc ...

The left hand side will contain your functions.

Veedrac
  • 58,273
  • 15
  • 112
  • 169
  • Am getting these errors --> File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.4/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/lib/python3.4/profile.py", line 589, in main() File "/usr/lib/python3.4/profile.py", line 575, in main code = compile(fp.read(), progname, 'exec') File "/home/****/eclipseworkspace/abcd.py", line 20 print '%r (%r, %r) %2.2f sec' % \ – ds_user Sep 21 '14 at 11:26
  • What happens if you just run `python file.py`? – Veedrac Sep 21 '14 at 11:29
  • Also, what does `python -V` say? It should start with "Python 2". – Veedrac Sep 21 '14 at 11:32
  • I have both 2.7 and 3.4.0 as part of my ubuntu 14 version. And so i changed the default version to Python 3.4.0 – ds_user Sep 21 '14 at 11:34
  • Right, so you'll need to run `python2 -m profile -s time file.py` as your code is Python 2. `print '%r (%r, %r) %2.2f sec'` is Python 2 syntax. – Veedrac Sep 21 '14 at 11:37
  • I am getting errors.. Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/usr/lib/python2.7/cProfile.py", line 199, in main() File "/usr/lib/python2.7/cProfile.py", line 192, in main runctx(code, globs, None, options.outfile, options.sort) File "/usr/lib/python2.7/cProfile.py", line 49, in runctx prof = prof.runctx(statement, globals, locals) – ds_user Sep 21 '14 at 11:41
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/61622/discussion-between-jeeva-and-veedrac). – ds_user Sep 21 '14 at 11:41
6

first i suggest use profilers module or timeit for this aim .timeit provides a simple way to time small bits of Python code !

To profile a function that takes a single argument, you can do:

import cProfile
import re
cProfile.run('re.compile("foo|bar")')

Also you can use a Decorator like this that allows you to measure the execution times of dedicated methods :

import time                                                

def measure_time(f):

  def timed(*args, **kw):
    ts = time.time()
    result = f(*args, **kw)
    te = time.time()

    print '%r (%r, %r) %2.2f sec' % \
          (f.__name__, args, kw, te-ts)
    return result

return timed

You can use it like this :

  @measure_time
  def foo():
        #content of function

note that f.__name__ return the name of function ! (in this case 'foo')

Mazdak
  • 105,000
  • 18
  • 159
  • 188
  • Sorry, i have never done this before. So, i have to add this measure_time function in the very starting of my program and add @measure_time before every function of mine? Am i right? – ds_user Sep 21 '14 at 11:18
  • Is it possible to use profile or timeit for the whole program? And does that give me time taken by every function or i have to use them to find the time taken by one function once? – ds_user Sep 21 '14 at 11:19
  • 1
    you need define the function in your `.py` and use `@measure_time` be fore any function for find those runtime ! – Mazdak Sep 21 '14 at 11:20
  • its best to use profile and you can use it for all of your code – Mazdak Sep 21 '14 at 11:24
  • 1
    To profile a function that takes a single argument, you can do: import cProfile `import re cProfile.run('re.compile("foo|bar")')` – Mazdak Sep 21 '14 at 11:25
  • read the more detail on wiki : https://docs.python.org/2/library/profile.html#the-python-profilers – Mazdak Sep 21 '14 at 11:26
  • The problem is i am looking for something which will give me the time taken by each function in my module like func1 - time taken, func2 - time taken.. like this.. Thats why i am trying to understand if i can use profiler or timeit to get like this. Because, profiler or timeit looks for each function at a time, not a whole module at a time. – ds_user Sep 21 '14 at 11:29
  • ok so as i write in answer you can use `re.comple` for add your function names – Mazdak Sep 21 '14 at 11:31