Python stats: how do I write it to a (human readable) file

Question

I am using Python's hotshot profiler: http://docs.python.org/2/library/hotshot.html

It shows how to print the stats:

stats.print_stats(20)

But how do I get that into a file? I'm not sure how to get at the information so I can write it to a file using write().

EDIT:

I'd like the same easily readable result as is printed when it's done this way:

stats = hotshot.stats.load("stones.prof")
stats.strip_dirs()
stats.sort_stats('time', 'calls')
stats.print_stats(20)

So it looks like this:

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1    3.295    3.295   10.090   10.090 pystone.py:79(Proc0)

(So not for it to look like when I open stones.prof)

score 49 · Answer 1 · edited Aug 02 '18 at 18:08

49

Stats takes an optional 'stream' argument. Simply open a file and pass the open file object to the Stats constructor as shown below. From that point any call to print_stats() will output to the stream you passed into the constructor. Hope this helps. :)

with open('path/to/output', 'w') as stream:
    stats = pstats.Stats('path/to/input', stream=stream)
    stats.print_stats()

edited Aug 02 '18 at 18:08

Utku

2,025
22
42

answered May 08 '13 at 17:34

Raos

511
4
6

1

Follow code works for me: import StringIO;stream = StringIO.StringIO(); stats = pstats.Stats(fname, stream=stream); stats.print_stats(); stream.seek(0); print stream.read() – moylop260 Feb 23 '16 at 05:00

score 3 · Answer 2 · answered Apr 13 '13 at 19:25

3

How about output redirection?

import sys
import pstats
sys.stdout = open('readable.profile', 'w')
p = pstats.Stats('input.profile')
p.print_stats()

answered Apr 13 '13 at 19:25

m3wolf

114
4

Unfortunately `p.print_stats()` does not use `sys.stdout` for printing. If you try this you get an empty file and the info printed to console instead of the file. – Cole Oct 02 '20 at 04:02

user984003 · Accepted Answer · 2012-11-24T10:11:54.863

I ended up rewriting the print_stats() function, starting with copying it from pstats.py. It returns a string, which can then be written to a file. I have not tested every if-else loop, just that it works in the examples that I needed it for. I left the original lines commented out. I've left the variable names the same although it isn't really "self" that the function is using anymore.

stats = hotshot.stats.load("stones.prof")
stats.strip_dirs()
stats.sort_stats('time', 'calls')
readable_str = xprint_stats(stats, 20)

import pstats
def xprint_stats(self, *amount):
    x = ""
    for filename in self.files:
        x += " " + filename
    #if self.files: print >> self.stream
    # ?
    indent = ' ' * 8
    for func in self.top_level:
        #print >> self.stream, indent, xfunc_get_function_name(func)
        x += indent + pstats.func_get_function_name(func)

    #print >> self.stream, indent, self.total_calls, "function calls",
    x +=  indent + str(self.total_calls) + " function calls" + " "
    if self.total_calls != self.prim_calls:
        #print >> self.stream, "(%d primitive calls)" % self.prim_calls,
        x += "(%d primitive calls)" % self.prim_calls + " "
    #print >> self.stream, "in %.3f seconds" % self.total_tt
    #print >> self.stream
    x +=  "in %.3f seconds" % self.total_tt + "\n"
    #width, list = stats.get_print_list(amount)
    msg, width, list = xget_print_list(stats, amount)
    x += msg

    if list:
        #self.print_title()
        x += "\n" + '   ncalls  tottime  percall  cumtime  percall filename:lineno(function)'
        x += "\n"
        for func in list:
            #self.print_line(func)
            x +=  xprint_line(self, func) + "\n"
#        print >> self.stream
#        print >> self.stream
    #return self
    return x

def xprint_line(self, func):  
    x = ""
    cc, nc, tt, ct, callers = self.stats[func]
    c = str(nc)
    if nc != cc:
        c = c + '/' + str(cc)
#    print >> self.stream, c.rjust(9),
#    print >> self.stream, f8(tt),
    x +=  c.rjust(9) + " "
    x +=  pstats.f8(tt) + " "
    if nc == 0:
        #print >> self.stream, ' '*8,
       x +=  ' '*8 
    else:
        #print >> self.stream, f8(float(tt)/nc),
        x +=  pstats.f8(float(tt)/nc) + " "
    #print >> self.stream, f8(ct),
    x +=  pstats.f8(ct) + " "
    if cc == 0:
        #print >> self.stream, ' '*8,
        x +=  ' '*8
    else:
        #print >> self.stream, f8(float(ct)/cc),
        x +=   pstats.f8(float(ct)/cc) + " "
    #print >> self.stream, func_std_string(func)
    x +=  pstats.func_std_string(func) + " "
    return x

def xget_print_list(self, sel_list):
    width = self.max_name_len
    if self.fcn_list:
        stat_list = self.fcn_list[:]
        msg = "   Ordered by: " + self.sort_type + '\n'
    else:
        stat_list = self.stats.keys()
        msg = "   Random listing order was used\n"

    for selection in sel_list:
        stat_list, msg = self.eval_print_amount(selection, stat_list, msg)

    count = len(stat_list)

    if not stat_list:
        return 0, stat_list
    #print >> self.stream, msg
    if count < len(self.stats):
        width = 0
        for func in stat_list:
            if  len(pstats.func_std_string(func)) > width:
                width = len(pstats.func_std_string(func))
    #return width+2, stat_list
    return msg, width+2, stat_list

moylop260 · Answer 4 · 2016-02-25T17:45:00.933

You can use the library: pstats_print2list https://pypi.python.org/pypi/pstats_print2list

pip install pstats_print2list And usage:

from pstats_print2list import get_pstats_print2list, print_pstats_list
fname_stats = 'my_profiling_out.stats'
pstats_list = get_pstats_print2list(
    os.path.expanduser(fname_stats),
    filter_fnames=['myfile1.py', 'myfile2.py', 'root_path1'],
    exclude_fnames=['dontshow.py', 'path_dont_show'],
    sort='cumulative',
    limit=5,
)
print_pstats_list(pstats_list)

score 0 · Answer 5 · answered Jul 06 '21 at 09:50

For those looking for another solution, where we can put the string data to a txt file. The code below will use two functions to add two numbers and then make the Profile data and finally writes it to an out.txt file.

main.py

import cProfile
import time
import pstats
from io import StringIO

def add_slow(a, b):
    time.sleep(0.5)
    return a+b

def add_fast(a, b):
    return a+b

prof = cProfile.Profile()

def main_func():
    arr = []
    prof.enable()
    for i in range(10):
        if i%2==0:
            arr.append(add_slow(i,i))
        else:
            arr.append(add_fast(i,i))
    prof.disable()
    #prof.print_stats(sort='time')
    prof.dump_stats("main_funcs.prof")
    return arr

main_func()
stream = StringIO();
stats = pstats.Stats("main_funcs.prof", stream=stream); 
stats.print_stats()
stream.seek(0)
print(16*'=',"RESULTS",16*'=')
data = stream.read()
print(data)
myText = open('out.txt', 'w')
myText.write(data)
myText.close()

score -2 · Answer 6 · answered Nov 23 '12 at 16:15

-2

stats.dump_stats(path/to/file)

answered Nov 23 '12 at 16:15

Katriel

120,462
19
136
170

2

I tried it and that file isn't readable. I'll edit my question to make it clearer. – user984003 Nov 23 '12 at 16:53
2

You could always just [redirect stdout to a file](http://stackoverflow.com/questions/6796492/python-temporarily-redirect-stdout-stderr). – Katriel Nov 23 '12 at 17:09

Python stats: how do I write it to a (human readable) file

6 Answers6

main.py