20

Is there any tool/library through which the list of methods/functions called within another methods/functions can be listed?

For example: If that tool or library runs for below method

def calculate(a: int, b: int, operator: Operator):
    if operator == Operator.add:
        add(a, b)
    elif operator == Operator.subtract
        subtract(a, b)

then it should return

1. add
2. subtract

This question is almost same as this one but it's for Java.

This is basically same as what PyCharm does for Find Usage. Thanks!

Stephen C
  • 698,415
  • 94
  • 811
  • 1,216
Pallav Jha
  • 3,409
  • 3
  • 29
  • 52
  • 1
    add and subtract are function calls rather than function definition @Stack – mad_ Aug 17 '18 at 19:25
  • @Stack still answering the wrong question – Jon Kiparsky Aug 17 '18 at 19:27
  • @Stack Nope That answers who calls add rather than OP's question. So No. – mad_ Aug 17 '18 at 19:27
  • Try this [lib](https://docs.python.org/3/library/inspect.html) for retrieving the interpreter stack. – Lev Zakharov Aug 17 '18 at 19:27
  • @Stack That would require change in the code :) – Pallav Jha Aug 17 '18 at 19:28
  • 2
    You can use [`dis`](https://docs.python.org/3/library/dis.html) to disassemble the function. Look for the `CALL_FUNCTION` bytecodes. You'll have to work backwards from them to get the function they're calling (which won't always be there). – Patrick Haugh Aug 17 '18 at 19:45
  • @PatrickHaugh `dis` is the library I was trying to remember. I suspect that you have a correct answer there if you want to write it up. – Jon Kiparsky Aug 17 '18 at 19:48
  • @PatrickHaugh Well ok I figured out why you didn't want to implement it yourself, took me 2h, but thanks for pointing out `dis` ... I wasn't aware of it until now. – Fabian N. Aug 17 '18 at 23:39
  • @ObiWan-PallavJha My answer below fails as noted, but it does answer the question "does such a library or tool exist?". The answer is yes, it is called `dis`. And to fully implement a general solution using that tool would, I think, require work approximately equivalent to implementing python itself. – Jon Kiparsky Aug 18 '18 at 00:26
  • So to sum it up, _there will be an easy to use tool / lib / python package_ that will do _exactly what you wantend_ in the near future (= like mid next week or so =D – Fabian N. Aug 18 '18 at 01:28
  • @JonKiparsky I got it working for Python2 too, cheers – Fabian N. Aug 18 '18 at 11:27

2 Answers2

12

This seems to do the work:

import dis
def list_func_calls(fn):
    funcs = []
    bytecode = dis.Bytecode(fn)
    instrs = list(reversed([instr for instr in bytecode]))
    for (ix, instr) in enumerate(instrs):
        if instr.opname=="CALL_FUNCTION":
            load_func_instr = instrs[ix + instr.arg + 1]
            funcs.append(load_func_instr.argval)

    return ["%d. %s" % (ix, funcname) for (ix, funcname) in enumerate(reversed(funcs), 1)]

Example:

>>> list_func_calls(calculate)
['1. add', '2. subtract']

What's happening here is:

  1. we make a Bytecode object of the function
  2. we reverse the list of instructions, since the function name will follow the function call
  3. we step through the list, and for each CALL_FUNCTION instruction,

  4. we use the instructions arg parameter to tell us how many arguments we're getting

  5. we look one past that to find the instruction that loads the function we're calling

  6. we add that function's name (instr.argval) to a list which we then reverse, enumerate, and return in the requested format

Note that since Python 3.6, there are three CALL_FUNCTION instructions, so you'll have to check the documentation to extend this example to be fully functional with current python

Fabian N.
  • 3,807
  • 2
  • 23
  • 46
Jon Kiparsky
  • 7,499
  • 2
  • 23
  • 38
  • @FabianN's answer was posted as I was finishing this up, so I figured I'd go ahead and post it. Slightly different approach, which is interesting. – Jon Kiparsky Aug 17 '18 at 23:46
  • 1
    This fails if you have nested function calls, try it out with my test cases but maybe the use case of the op is limited to simple calls. – Fabian N. Aug 17 '18 at 23:54
  • @FabianN. Good point. Let me see if I can improve on it a little. Thanks! – Jon Kiparsky Aug 18 '18 at 00:10
  • 1
    I just found the next pitfall: list comprehension inside a function call, you can filter them with `"." not in entry.argval` – Fabian N. Aug 18 '18 at 00:15
  • My friend, I am pleased that you're enjoying this code so much. Thanks for encouraging me to improve it! – Jon Kiparsky Aug 18 '18 at 00:16
  • @FabianN. Considering the matter, I realize that the only way to make a really reliable version of this will involve constructing the stack and then more or less executing the code, which is more than I'm going to do. I think both answers satisfy the question ("Is there any tool/library through which the list of methods/functions called within another methods/functions can be listed?"), so I'm going to walk away from this before I accidentally implement python :) – Jon Kiparsky Aug 18 '18 at 00:23
  • I just realized that I can skip the whole filter step at the beginning so I think my solution approaches more or less at an optimum. Actually, I think its pretty stable now. Just walking through the stack in reverse order and skipping everything that contians a "." (which fortunally isn't allowed in normal function names) seems to do the trick. And unless someone provides a counter example I will consider it to be a *general solution* :P – Fabian N. Aug 18 '18 at 00:27
  • It seems that this function is faulty. When the arguments passed to a function are expressions (e.g. `func(np.exp(a), b)`), the result I get is `None`. – Sandu Ursu May 06 '20 at 00:27
  • @SanduUrsu Yes, it's likely that a piece of code whipped up as an example has edge cases that are failing. – Jon Kiparsky May 06 '20 at 15:28
  • Note that this solution only helps in the obvious call cases, like a function in the global scope of the function, but when calling an instance method, or a method from an imported module, this is not so simple. More instructions are called, for loading the different wrappers of the function, and when chaining function calls. I think it would be very hard to track all function calls, and impossible when considering functions called from arguments. – Uriya Harpeness Apr 21 '21 at 06:43
6

Update: added compatibility for Python2.7
Tested and confirmed working with Python2.7, Python3.5 and Python3.6


Credit for pointing out dis goes to Patrick Haugh¹
Implementation (parsing of the dis output) is my own:


Setup:

import dis
import sys
from contextlib import contextmanager

# setup test environment
def a(_,__):
    pass

def b(_,__,___):
    pass

def c(_):
    pass

def g():
    pass 

d = 4

def test(flag):
    e = c

    if flag:
        a(a(b,c), [l for l in g(1, x=2)])
    else:
        b(a, int(flag), c(e))

    d = d + 1


def calculate(a, b, operator):
    if operator == Operator.add:
        add(a, b)
    elif operator == Operator.subtract:
        subtract(a, b)

class Operator(object):
    add = "add"
    subtract = "subtract"

Python 2/3 compatibility:

class AttrDict(dict):
    def __init__(self, *args, **kwargs):
        super(AttrDict, self).__init__(*args, **kwargs)
        self.__dict__ = self

@contextmanager # https://stackoverflow.com/a/12111817/2422125
def captureStdOut(output):
    stdout = sys.stdout
    sys.stdout = output
    try:
        yield
    finally:
        sys.stdout = stdout


""" for Python <3.4 """
def get_instructions(func):
    import StringIO

    out = StringIO.StringIO()
    with captureStdOut(out):
        dis.dis(func)

    return [AttrDict({
               'opname': i[16:36].strip(),
               'arg': int(i[37:42].strip() or 0),
               'argval': i[44:-1].strip()
           }) for i in out.getvalue().split("\n")]


if sys.version_info < (3, 4):
    dis.get_instructions = get_instructions
    import __builtin__ as builtin
else:
    import builtins as builtin

Code:

def get_function_calls(func, built_ins=False):
    # the used instructions
    ins = list(dis.get_instructions(func))[::-1]

    # dict for function names (so they are unique)
    names = {}

    # go through call stack
    for i, inst in list(enumerate(ins))[::-1]:
        # find last CALL_FUNCTION
        if inst.opname[:13] == "CALL_FUNCTION":

            # function takes ins[i].arg number of arguments
            ep = i + inst.arg + (2 if inst.opname[13:16] == "_KW" else 1)

            # parse argument list (Python2)
            if inst.arg == 257:
                k = i+1
                while k < len(ins) and ins[k].opname != "BUILD_LIST":
                    k += 1

                ep = k-1

            # LOAD that loaded this function
            entry = ins[ep]

            # ignore list comprehensions / ...
            name = str(entry.argval)
            if "." not in name and entry.opname == "LOAD_GLOBAL" and (built_ins or not hasattr(builtin, name)):
                # save name of this function
                names[name] = True

            # reduce this CALL_FUNCTION and all its paramters to one entry
            ins = ins[:i] + [entry] + ins[ep + 1:]

    return sorted(list(names.keys()))

Output:

> print(get_function_calls(test))
> ['a', 'b', 'c', 'g']

> print(get_function_calls(test, built_ins=True))
> ['a', 'b', 'c', 'g', 'int']

> print(get_function_calls(calculate))
> ['add', 'subtract']

¹As Patrick Haugh's comment about dis is over 2h old I consider this one free for taking...

Fabian N.
  • 3,807
  • 2
  • 23
  • 46
  • 1
    Nice approach, but will currently break on functions with keywoard arguments for Python 3.6+. They introduced `CALL_FUNCTION_KW` (and `CALL_FUNCTION_EX`). Worth noting may also be that it will also catch things like class instantiation and builtins like `dict()` and `tuple()`, `int()`... technically not wrong but probably not what the user might think of. – Darkonaut Aug 20 '18 at 01:35
  • @Darkonaut good point, I will add a switch to ignore class instantiation and the like, parsing `CALL_FUNCTION_EX` will get funny ... eta sometime today ... I think – Fabian N. Aug 20 '18 at 04:33
  • @Darkonaut I got keyword arguments working and added a flag to disable inbuilt functions but class constructors have to stay for now (no Idea on how to filter them without eval) – Fabian N. Aug 20 '18 at 05:26
  • @PaulRooney thanks for pointing that out but as my only input was Patrick Haugh's comment "[...]You can use dis to disassemble the function[...]" I didn't see it appropriate. The main work was to parse the output, I just added a little note at the top as basic courtesy. – Fabian N. Aug 20 '18 at 05:40
  • Still needs some refinement. You didn't define `builtin` here before `hasattr(builtin, name)` so you get `NameError ` when you set `built_ins=False`. If you set `built_ins=True` it won't include builtins with keywoard arguments, e.g. `enumerate([1,2], start=1)`. – Darkonaut Aug 20 '18 at 11:58
  • @Darkonaut sorry but I can't reproduce the first part, `builtin` is defined by `import ... as builtin` at the end of the "Python<3.4" compatibility block... @the second part hm that indeed needs some refinement. – Fabian N. Aug 20 '18 at 12:44
  • Ah there you have it ;), didn't look at that part because I thought the whole block is just relevant for Python<3.4 – Darkonaut Aug 20 '18 at 12:50
  • @Darkonaut could you reproduce the bug after including the missing import? (I edited the answer to make it more clear that the second code block is [partly] required even if you are using Python >3.4) – Fabian N. Aug 20 '18 at 16:29
  • No no, that part worked fine with the included import. – Darkonaut Aug 20 '18 at 17:18
  • Appears to give incomplete results for a function of mine. – Sandu Ursu May 06 '20 at 00:23