50

I'd like to profile a method of a function in Python, using cProfile. I tried the following:

import cProfile as profile

# Inside the class method...
profile.run("self.myMethod()", "output_file")

But it does not work. How can I call a self.method with "run"?

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
  • 1
    Possible duplicate of [How can you profile a Python script?](https://stackoverflow.com/questions/582336/how-can-you-profile-a-python-script) – moshevi Sep 11 '19 at 19:58

5 Answers5

73

EDIT: Sorry, didn't realise that the profile call was in a class method.

run just tries to exec the string you pass it. If self isn't bound to anything in the scope of the profiler you are using, you can't use it in run! Use the runctx method to pass in the local and global variables in the scope of the call to the profiler:

>>> import time
>>> import cProfile as profile
>>> class Foo(object):
...     def bar(self):
...             profile.runctx('self.baz()', globals(), locals())
...
...     def baz(self):
...             time.sleep(1)
...             print 'slept'
...             time.sleep(2)
...
>>> foo = Foo()
>>> foo.bar()
slept
         5 function calls in 2.999 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    2.999    2.999 <stdin>:5(baz)
        1    0.000    0.000    2.999    2.999 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        2    2.999    1.499    2.999    1.499 {time.sleep}

Notice the last line: time.sleep is what's taking up the time.

Katriel
  • 120,462
  • 19
  • 136
  • 170
  • This just doesn't work on Python 2.6.4 on Mac OS X. I get the error NameError: name 'foo' is not defined –  Dec 20 '10 at 18:28
  • 1
    @user: sorry, my bad. Edited. – Katriel Dec 20 '10 at 18:29
  • 2
    How can I make the profiler "dig deeper", i.e. not just say that all the time was spent in some function of a module but recursively delve into functions called in that module? –  Dec 20 '10 at 18:51
  • What do you mean? The profiling times code starting from the call of the method until the return value. It should give results for every method called in that module. You should check out my answer, it will give you a nice graph of what the result is – Falmarri Dec 20 '10 at 20:32
  • 2
    **Brilliantness!** Passing `cProfile.runctx()` the current `globals()` and `locals()` permits arbitrary methods to be profiled – exactly as advertised. Thanks for this, [katrielalex](https://stackoverflow.com/users/398968/katrielalex). – Cecil Curry Jun 02 '16 at 01:55
  • Does `runctx` return values that the profiled method did return? – Tomáš Zato Nov 21 '16 at 01:39
  • This is very helpful! Just to add this small point: if you want to profile with arguments, just include them in the string as well. For example, if we have two arguments in the baz method, the new profile method will be like this: def bar (self, argument1, argument2) and then the statement will become: profile.runctx('self.baz(argument1, argument2)', globals(), locals()) – Brevis Iunius Jun 05 '19 at 16:05
31

Use the profilehooks decorator

http://pypi.python.org/pypi/profilehooks

Falmarri
  • 47,727
  • 41
  • 151
  • 191
  • Handy, but doesn't work on Python 3, though 2to3 fixes that. :) – Adam Parkin May 31 '12 at 22:21
  • 1
    Profilehooks now supports Python 3. See here: pypi.python.org/pypi/profilehooks – MackM May 12 '15 at 21:43
  • 16
    **No.** Don't install heavyweight dependencies. Just call `cProfile.runctx('self.myMethod()', globals(), locals(), output_file)`, as [katrielalex](https://stackoverflow.com/users/398968/katrielalex) suggests in what _should_ have been the [accepted answer](https://stackoverflow.com/a/4492582/2809027). – Cecil Curry Jun 02 '16 at 01:53
  • profilehooks works perfectly to profile Tkinter applications. With the cProfile approach, you're having it directly execute a function, which if the function is supposed to be a callback to a Tkinter.Button, may not be so easy w/o restructuring code. With profilehooks, importing the `profile` decorator and applying it to a specific callback lets you test just that callback only when the button is clicked. – Kumba Jul 11 '17 at 06:21
4

If your function under profile returns value(s), you need to change the excellent answer from @katrielalex slightly:

...             profile.runctx('val = self.baz()', globals(), locals())
...             print locals()['val']
sqqqrly
  • 141
  • 1
  • 3
4
  import cProfile
  p = cProfile.Profile()
  p.runcall(self.myMethod)
  p.print_stats()

The Profile class is documented here.

Julius Kunze
  • 1,035
  • 11
  • 20
1

I wouldn't recommend profiling a single routine, because that implies knowing in advance there's a problem there.

A fundamental aspect of performance problems is they're sneaky. They're not where you think they are, because if they were you would have solved them already.

It's better to run the whole program with a realistic workload and let the profiling technique tell you where the problems are.

Here's an example where profiling finds the problem, and it is not where expected.

Community
  • 1
  • 1
Mike Dunlavey
  • 40,059
  • 14
  • 91
  • 135
  • 4
    All I want to do is run a profiler on this top level routine and have the profiler actually delve in to the functions instead of just saying all the top was spend in some top level routine, since that is completely useless. –  Dec 20 '10 at 18:55
  • @user248237: Right. Just do what I said in the link. No profiler will give you better information. – Mike Dunlavey Dec 20 '10 at 20:56
  • @Mike Dunlavey: Not necessarily. While profiling the entire code, you may then have determined that the bottleneck appears in one function. In which case, it would make sense to then profile that one function to see what's going on there.. – Gino Sep 17 '17 at 11:11
  • @Gino: What's a bottleneck? Lots of people talk about bottlenecks as if "everybody" knows what they are. If a mega-line C# application spends 50% of its time reading dlls to extract string attributes so that they can be translated for the current geography, when you see by looking at the strings that it doesn't need to, is that a bottleneck? Would a profiler find it? (No) Would random-pausing find it? (Yes) Random-pausing will locate any bottleneck that any profiler will, and more that it won't. – Mike Dunlavey Sep 17 '17 at 20:28
  • @Mike Dunlavey: Frankly, I'm not sure what your point is. The basic assumption is that the programmer understands the code well enough to discern what is relevant and what is not. When I say a bottleneck appears a function, one obvious way that this can happen is if I run a profiler and the profiler tells me that 90% of the time is spent in function A. So function A is a bottleneck that I want to drill down into. And I want to know which lines of function A are taking up most of the time.. – Gino Sep 18 '17 at 05:54
  • @Gino: Is that 90% exclusive or inclusive of called routines? Does it include blocked time like I/O, or only CPU time? Certainly the top-level function spends all the time, but that's kind of pointless. Some profilers tell you the "hot path", but it's [*easy for performance problems to hide from that*](https://stackoverflow.com/a/25870103/23771). What if the problem isn't in a particular function, but in a design decision, causing a bunch of different functions to be called unnecessarily? And what if there are 10 programmers, not one? – Mike Dunlavey Sep 18 '17 at 15:14
  • @MikeDunlavey: I think you may be unnecessarily complicating the issue. Again, the ability to profile a single function is useful because it allows you to see exactly where the possible bottlenecks are. You mention bad design decisions as a source of performance problems, but that is really outside the purview of this stackoverflow question. – Gino Sep 19 '17 at 04:47
  • @Gino: Well if you want maximum performance, you need to go after the opportunities that are hiding. Anybody can get the easy stuff, but if you have a competitor, the winner is the one who also gets the hard stuff. – Mike Dunlavey Sep 19 '17 at 17:30