Filter profiler files with maximum cumulative time greater then X

Question

Say, I create binary profile files with the similar code:

import cProfile
profiler = cProfile.Profile()
...
profiler.dump_stats( 'file1.profile' )

I have many profile files. I need to find all files with maximum cumulative time greater then x seconds. Are there approaches to do that?

If your unstated goal is to find out what you can do to make the code faster, [*see this answer*](http://stackoverflow.com/a/4299378/23771). — Mike Dunlavey, May 22 '14 at 11:55

Padraic Cunningham · Accepted Answer · 2014-05-22T13:12:25.953

Use pstats to get the total times from each file.

import cProfile
import pstats
import glob
import os

cProfile.run('[str(x) for x in range(10000000)]', 'restats.profile')
cProfile.run('[str(x) for x in range(10000000)]', 'restats1.profile')
cProfile.run('[str(x) for x in range(100000)]', 'restats2.profile')
def prof_times(dir):
    os.chdir(dir)
    max_sec=1
    over_max=[]
    for f in glob.glob("*.profile"):
        p = pstats.Stats(f)
        p.print_stats()
        if p.total_tt > max_sec:
            over_max.append(f)
    return over_max
print prof_times("/home/test")

Thu May 22 13:11:28 2014    restats1.profile

         3 function calls in 2.779 seconds

   Random listing order was used

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    2.678    2.678    2.779    2.779 <string>:1(<module>)
        1    0.100    0.100    0.100    0.100 {range}


Thu May 22 13:11:26 2014    restats.profile

         3 function calls in 2.885 seconds

   Random listing order was used

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    2.685    2.685    2.885    2.885 <string>:1(<module>)
        1    0.201    0.201    0.201    0.201 {range}


Thu May 22 13:11:28 2014    restats2.profile

         3 function calls in 0.028 seconds

   Random listing order was used

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.027    0.027    0.028    0.028 <string>:1(<module>)
        1    0.001    0.001    0.001    0.001 {range}


['restats1.profile', 'restats.profile']

Filter profiler files with maximum cumulative time greater then X

1 Answers1