How to add suffix operations for all python functions?

Question

I have a a collection of functions inside a file myfile.py and I would like to add a timer print automatically for each of them: (have around 700 functions)

def myfunXXX() 
   mytime.start()

   mytime.end() 

mytime.start(): to= time.time()
mytime.end():   print(time.time() - t0)

Current solutions are :

Add a decorator manually to all the functions.
Add the code snippet to all the functions

Is there a way to "hack" Python and the function base definition to add those 2 functions before executing and at execution end for a set of Python functions ?

Maybe using AST and injecting some string code ?

I need to add the decorators to the 500 functions in the code. Prefer something automatic — quantCode, Jan 21 '22 at 01:31
You could probably regex search-and-replace the decorator onto the function declarations. — Mike Clark, Jan 21 '22 at 01:33
I was about to suggest something like that. Much more preferable than hacking Python's import logic. — Daniel Walker, Jan 21 '22 at 01:34
[python - Viewing all defined variables - Stack Overflow](https://stackoverflow.com/questions/633127/viewing-all-defined-variables) ? Then check which ones are functions and patch. — user202729, Jan 21 '22 at 01:36
You could probably iterate over the functions in the file [using reflection](https://stackoverflow.com/a/139198/312407) and create proxy functions. But then you'd have to update the code which uses the functions to pull the functions from the proxy module instead, which might also be a hassle. — Mike Clark, Jan 21 '22 at 01:37
As others suggested, this is a job that a profiler already does. See [this page](https://docs.python.org/3/library/profile.html) to learn more (in particular about `cProfile`). tl;dr: `python -m cProfile -s cumulative myfile.py` — Amadan, Jan 21 '22 at 01:54

Bharel · Accepted Answer · 2022-01-24T14:13:34.110

2

sys.setprofile is meant exactly for this.

Make sure you filter according to frame.f_code.co_filename to only stop on the wanted files.

On call event start the timer, on return stop.

Make sure you use time.perf_counter as the timer. The others are less reliable.

Keep in mind, as any other profiler that checks all functions in a module, especially one written in Python, it will slow down your code.

Example:

import sys, time


def create_profile_function(filename):
    _active_frames = {}

    def profile_func(frame, event, arg):
        if event == "call" and frame.f_code.co_filename == filename:
            _active_frames[frame] = time.perf_counter()
        elif event == "return" and frame in _active_frames:
            print(frame.f_code.co_name,
                  time.perf_counter() - _active_frames.pop(frame))

    return profile_func

sys.setprofile(create_profile_function(__file__))

def test():
    print("rawr")
    pass

test()

Output:

rawr
test 0.0005169999785721302

edited Jan 24 '22 at 14:13

answered Jan 21 '22 at 01:46

Bharel

23,672
5
40
80

I still doubt they really need such custom profiling code instead of just using an existing profiler, but interesting anyway. – Kelly Bundy Jan 21 '22 at 01:53
@KellyBundy he might need modifications or printing on-demand, no clue :-) – Bharel Jan 21 '22 at 09:52
This is not necessary profiling, more getting the time at exit for a list of 500 functions (from one module .py) – quantCode Jan 22 '22 at 05:36
@quantCode my answer works nonetheless – Bharel Jan 22 '22 at 08:18
1

thanks, Can you write a complete and minimal example ? After will approve. – quantCode Jan 24 '22 at 02:34
@quantCode added. – Bharel Jan 24 '22 at 14:16
sure, thx. How to execute only on specific function name of the file myfile.py ? – quantCode Jan 25 '22 at 00:20
`frame.f_code.co_filename == "myfile.py" and frame.f_code.co_name == "func"` – Bharel Jan 25 '22 at 00:25

score -1 · Answer 2 · answered Jan 21 '22 at 01:48

One solution if you're willing to modify where you call these functions is to do something like this:

import time
def wrapper(function,args):
    start = time.time()
    function(args)
    end = time.time()
    print(end-start)

I know you don't want to add decorators because you have a lot of functions to add them to. If you're willing to put your functions in a class you might find this answer or this answer useful on automatically adding decorators to classes.

score -1 · Answer 3 · answered Jan 21 '22 at 02:06

Decorators that Patch Classes is one "hack" that comes to mind.

def _task(cls):
   og_attrs = cls.__getattribute__
   def new_getattrs(self, name):
       return og_attrs(self, name)
   cls.__getattribute__ = new_getattrs
   return cls 

@_task # mass change in class method's behavior
class A:
    def reborn_function(self):
        pass

Your whole issue might not go away at once. But this is the closest thing you have to be backward compatible and elegant. Ast will be a wrong move if you are worried about API changes or backward compatibility.

How to add suffix operations for all python functions?

3 Answers3