73

I am using cProfile on a module named bot4CA.py so in the console I type: 

python -m cProfile -o thing.txt bot4CA.py

After the module runs and exits, it creates a file named thing.txt and when I open it, there is some information there, and the rest is a jumble of characters instead of a neatly organized file of data which is what I want. Does any one know how to use cProfile and end up with neatly organized table of data like when using it normally on command line, except in a file? Here's an example of some of the data in the .txt file:

{(   s)   build\bdist.win32\egg\colorama\winterm.pyi'   t      reset_all(   i   i   gpàÂs% ?geOÙHÌœE?{(   s-   build\bdist.win32\egg\colorama\ansitowin32.pyi¥

What I really want is what happens when you invoke cProfile.run() which results in a neatly organized table printed showing the execution times of all the functions except instead of printed, saved in a file as this program is fairly large and runs a lot of functions.

joseph
  • 1,021
  • 2
  • 9
  • 11

3 Answers3

93

You should use the pstats module to parse this file and extract information in user-friendly format from it. For example:

import pstats
p = pstats.Stats('thing.txt')
p.sort_stats('cumulative').print_stats(10)

It's all in the documentation, of course. Go over the "instant user's manual" in there, it explains everything.

Eli Bendersky
  • 263,248
  • 89
  • 350
  • 412
  • 3
    How would I be able to do that using command line as with: python -m cProfile -o thing.txt bot4CA.py. The reason why is because cProfile doesn't seem to want to work with running entire scripts. – joseph Nov 27 '11 at 03:05
  • well, now I have a second problem, this program uses twisted and the data only goes as deep as twisted's main loop, rather than the methods that twisted calls in the script – joseph Nov 27 '11 at 03:14
  • @joseph: the linked doc explains command-line usage, although it's relatively limited in the reporting options it offers – Eli Bendersky Nov 27 '11 at 03:28
  • What about the twisted problem where it only gives the times for the twisted main loop or something on that level, instead of the methods that the twisted Framework calls like dataRecieved? – joseph Nov 27 '11 at 05:02
  • 2
    @joseph: I have no idea - never tried profiling Twisted. You can open a separate question or ask in some Twisted forums/lists – Eli Bendersky Nov 27 '11 at 05:54
  • 1
    Unfortunately, there's nothing in the linked documentation that describes what the file format is, or what one can do with it... – Chris Dodd Jun 22 '15 at 23:36
  • @ChrisDodd: the format is internal, and specifically not documented. You're supposed to use `pstats` to work with it. This answers the question, which didn't inquire about the internal format, but rather how to make sense of it. – Eli Bendersky Jun 23 '15 at 03:26
  • Use dump_stats to save it to a file. – marsh Mar 24 '16 at 17:24
  • 1
    In the documentation I didn't see anything like this, but your code was very useful. Thanks! – KansaiRobot Oct 27 '22 at 05:49
69

to dump the stats driectly:

echo 'stats' | python3 -m pstats path/to/cprofile_output_file

pstats also has a shell

$ python3 -m pstats path/to/cprofile_output_file

within we can issue stats or sort commands like so:

$ python3 -m pstats path/to/cprofile_output_file
Welcome to the profile statistics browser.
prof.txt% sort cumtime
prof.txt% reverse
prof.txt% stats

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 63:1(<module>)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.print}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

prof.txt% ?

Documented commands (type help <topic>):
========================================
EOF  add  callees  callers  help  quit  read  reverse  sort  stats  strip

a micro-feature I enjoyed here is I get to reverse the sort order natively <3

echo -e 'sort cumtime\nreverse\nstats' | python3 -m pstats path/to/cprofile_output_file
ThorSummoner
  • 16,657
  • 15
  • 135
  • 147
  • 15
    This is a new answer on a very old question, but in my opinion it is much more helpful than the accepted answer because it matches the OP's command line cProfile usage. Annoyingly, the official docs don't demonstrate parsing the output file anywhere near demonstrating how to generate it. +1 – Rakurai Jan 08 '19 at 16:29
  • also: `echo -e 'sort tottime\nreverse\nstats' | python3 -m pstats <(python3 -m cProfile -o /dev/stdout ./myscript.py myargs)` – ThorSummoner Apr 27 '20 at 02:16
  • 1
    This was helpful. On my windows machine, the first part, the `echo` statement, is without quotes. Use (e.g.) `echo stats | python -m pstats path\to\profile_output` – Marc Meketon Jun 17 '20 at 17:38
9

The other answers are more powerful and flexible, but if you just want to get a quick output, use > instead of -o. That will save the report in plain text format.

python -m cProfile myscript.py > cprofile.txt
python -m cProfile bot4CA.py > thing.txt
wisbucky
  • 33,218
  • 10
  • 150
  • 101