Questions tagged [cprofile]

cProfile is a built-in Python module that describes the run time performance of a program, providing a variety of statistics.

cProfile is a built-in module suitable for profiling Python programs. It's designed as a faster drop-in replacement for the profile implementation. cProfile is implemented as C-Extension which provides less overhead than the pure Python implementation of profile, but it may not be available in all Python implementations.

It can be used in a Python file:

import cProfile
cProfile.run('insert_your_code_to_profile_here')

But also from the command-line, to profile a script:

python -m cProfile [-o output_file] [-s sort_order] (-m module | myscript.py)

Sources:

230 questions
110
votes
1 answer

What is the difference between tottime and cumtime on cProfile output?

I am profiling a python script main.py using cProfile with the following command: python -m cProfile -s tottime main.py The output I get is (only copy-pasted the top lines of the output): 10184337 function calls (10181667 primitive calls) in 13.597…
takahashi
  • 2,081
  • 3
  • 14
  • 24
104
votes
1 answer

Sort cProfile output by percall when profiling a Python script

I'm using python -m cProfile -s calls myscript.py python -m cProfile -s percall myscript.py does not work. The Python documentation says "Look in the Stats documentation for valid sort values.":…
Brandon O'Rourke
  • 24,165
  • 16
  • 57
  • 58
73
votes
3 answers

cProfile saving data to file causes jumbles of characters

I am using cProfile on a module named bot4CA.py so in the console I type: python -m cProfile -o thing.txt bot4CA.py After the module runs and exits, it creates a file named thing.txt and when I open it, there is some information there, and the…
joseph
  • 1,021
  • 2
  • 9
  • 11
56
votes
5 answers

Using cProfile results with KCacheGrind

I'm using cProfile to profile my Python program. Based upon this talk I was under the impression that KCacheGrind could parse and display the output from cProfile. However, when I go to import the file, KCacheGrind just displays an 'Unknown File…
Adam Luchjenbroers
  • 4,917
  • 2
  • 30
  • 35
53
votes
3 answers

Python multiprocess profiling

I want to profile a simple multi-process Python script. I tried this code: import multiprocessing import cProfile import time def worker(num): time.sleep(3) print 'Worker:', num if __name__ == '__main__': for i in range(5): p =…
barmaley
  • 1,027
  • 2
  • 11
  • 12
51
votes
1 answer

Python getting meaningful results from cProfile

I have a Python script in a file which takes just over 30 seconds to run. I am trying to profile it as I would like to cut down this time dramatically. I am trying to profile the script using cProfile, but essentially all it seems to be telling…
Bryce Thomas
  • 10,479
  • 26
  • 77
  • 126
50
votes
5 answers

profiling a method of a class in Python using cProfile?

I'd like to profile a method of a function in Python, using cProfile. I tried the following: import cProfile as profile # Inside the class method... profile.run("self.myMethod()", "output_file") But it does not work. How can I call a self.method…
user248237
47
votes
5 answers

Can I run line_profiler over a pytest test?

I have identified some long running pytest tests with py.test --durations=10 I would like to instrument one of those tests now with something like line_profiler or cprofile. I really want to get the profile data from the test itself as the pytest…
Atlas1j
  • 2,442
  • 3
  • 27
  • 35
41
votes
1 answer

Python cProfile results: two numbers for ncalls

I just recently began profiling a server application that I've been working on, trying to find out where some excess processor time is being spent and looking for ways to make things smoother. Overall, I think I've got a good hang of using cProfile…
Andrew Baird
  • 413
  • 4
  • 5
38
votes
3 answers

use PyCharm to view profiling data

Does anyone know if there is a way to open external .pstat files inside PyCharm for viewing? When I try to, I'm being asked to select the file type, but there doesn't seem to be the option for it (even those the view is used for internally…
Dmitry B.
  • 9,107
  • 3
  • 43
  • 64
36
votes
5 answers

saving cProfile results to readable external file

I am using cProfile try to profile my codes: pr = cProfile.Profile() pr.enable() my_func() # the code I want to profile pr.disable() pr.print_stats() However, the results are too long and cannot be fully displayed in the Spyder terminal (the…
Physicist
  • 2,848
  • 8
  • 33
  • 62
35
votes
1 answer

cProfile for Python does not recognize Function name

I have a function in an app called email which I want to profile. When I try to do something like this, it blows up from django.core.management import BaseCommand import cProfile class Command(BaseCommand): def handle(self, *args,…
Ben
  • 15,010
  • 11
  • 58
  • 90
31
votes
0 answers

>90% of the time is spent on method 'acquire' of 'thread.lock' objects

To identify the step that is using most of the computation time, I ran cProfile and got the following result: ncalls tottime percall cumtime percall filename:lineno(function) 1 0.014 0.014 216.025 216.025…
Sameeresque
  • 2,464
  • 1
  • 9
  • 22
29
votes
11 answers

How can I speed up fetching pages with urllib2 in python?

I have a script that fetches several web pages and parses the info. (An example can be seen at http://bluedevilbooks.com/search/?DEPT=MATH&CLASS=103&SEC=01 ) I ran cProfile on it, and as I assumed, urlopen takes up a lot of time. Is there a way to…
Parker
  • 8,539
  • 10
  • 69
  • 98
27
votes
3 answers

Why cProfile module doesn't work with unittest?

I would to use cProfile module to profile my unit tests. But when I run python -mcProfile mytest.py I got 'Ran 0 tests in 0.000s'. Here is the source code of mytest.py import unittest class TestBasic(unittest.TestCase): def testFoo(self): …
Ryan Ye
  • 3,159
  • 1
  • 22
  • 26
1
2 3
15 16