Does Python allow to parse a module and return all the functions/classes in it?

Question

I am curious if there is a way, beyond the parsing of the py file as text file, to retrieve what is actually inside the module.

For example; if I import a module, and want to know what the module contain, and its usage (parameters needed, return types and so on), is there a way to do this directly through Python, or I have to actually load the file, like I would do for a text file, and parse it manually?

I have plenty of modules that I would like to document, and something like this would save me time to go trough each module and check them one by one.

Lazy partial solution: `import some_module; print dir(some_module)` — Kevin, Jun 11 '15 at 18:59
Also have a look at this: https://wiki.python.org/moin/DocumentationTools — moooeeeep, Jun 11 '15 at 19:11

score 1 · Answer 1 · edited May 23 '17 at 11:53

You can use ast to properly parse all the module:

import inspect
import importlib
import ast


class ParseMod(ast.NodeVisitor):
    def visit_FuncDef(self,node):
        print("In func")
        print(node.name)
        print(node.args)
        self.generic_visit(node)

    def visit_ClassDef(self,node):
        print("in class")
        print(node.name)
        for n in node.body:
            print(n.name)
        self.generic_visit(node)

    def visit_Call(self, node):
        print("In call")
        print(node.func.id)
        print([ast.literal_eval(arg) for arg in node.args])
        self.generic_visit(node)


mod = "test2"
mod = importlib.import_module(mod)
p = ast.parse(inspect.getsource(mod))

par = ParseMod()

par.visit(p)

Input:

class Foobar(object):
    def func(self):
        return "func"


class Bar(object):
    def func2(self):
        return  "func2"

class Foo():
    def __init__(self):
        self.x = 1
        self.y = 2
        self.z = 3
    def func_foo(self):
        return "func_foo"


def some_func(a,b):
    """
    :param a: int
    :param b: int
    :return: int
    """
    return a,b

some_func(1,2)

Output:

in class
Foobar
func
in class
Bar
func2
in class
Foo
__init__
func_foo
In call
some_func
[1, 2]

There is a full list of all the available attributes and an explanation of each in the greentreesnakes docs.

If you just want to collect the function and class nodes:

import inspect
import importlib
import ast
mod = "test"
mod = importlib.import_module(mod)
p = ast.parse(inspect.getsource(mod))



classes = [node for node in p.body if isinstance(node, ast.ClassDef)]
funcs = [node for node in p.body if isinstance(node, ast.FunctionDef)]

print(classes)
print(funcs)
[<_ast.ClassDef object at 0x7f9bc7884190>, <_ast.ClassDef object at 0x7f9bc7884350>, <_ast.ClassDef object at 0x7f9bc7884510>]    
[<_ast.FunctionDef object at 0x7f89d3377c50>]

Then extract what you want from each:

for f in funcs:
    print(f.name)
    print([a.id for a in  f.args.args])
    print(ast.get_docstring(f))

some_func
['a', 'b']
:param a: int
:param b: int
:return: int

There are a couple more example usages here and here

@newbiez, no worries, it is hard to provide a more complete example as there are limitless ways to get data but ast is a very powerful tool when dealing with the source. I will add a few links to some other answers of mine that get various output — Padraic Cunningham, Jun 11 '15 at 20:59
Indeed; you went the extra mile to show the concept, I appreciate it, since I am a beginner at Python. Thanks! — , Jun 11 '15 at 21:05
@newbiez, no prob, I added a couple of links to answers using ast for different purposes — Padraic Cunningham, Jun 12 '15 at 00:22

score 0 · Answer 2 · answered Jun 11 '15 at 19:06

You can get a list of strings corresponding to every member of a module using the dir method:

import some_module
print dir(some_module)

This will get you all the classes and functions of the module, with two drawbacks:

it also gives you members that aren't classes or functions. For example, dir(Tkinter) contains VERTICAL and SUNKEN and OFF and INSERT, which are all just regular strings.
if the module contains any code at the file level, then that code will be executed. Generally not a big deal, but if you're trying to inspect a lot of modules, the time lost can pile up.

Tanks Kevin; these 2 issues may be bothersome; So far the few modules that I tried, returned what I needed, but I need more time to investigate — , Jun 11 '15 at 21:04

ali_m · Answer 3 · 2015-06-11T19:31:51.387

One way would be to use inspect.getmembers() to introspect the contents of the module:

import types
import inspect
import mymodule

# find classes, shove them in a {'name':object} dict
is_class = lambda x: isinstance(x, type)
classes = dict(inspect.getmembers(mymodule, predicate=is_class))

# find functions, shove them in a {'name':object} dict
is_func = lambda x: isinstance(x, (types.FunctionType, types.BuiltinFunctionType))
functions = dict(inspect.getmembers(mymodule, predicate=is_func))

# etc...

It sounds like what you're really looking for is a tool for automatically generating API documentation, of which there are several.

Thanks Ali; documentation automation is one of the purposes; I was also curious to know how to introspect in python modules. — , Jun 11 '15 at 21:02

Does Python allow to parse a module and return all the functions/classes in it?

3 Answers3