Python - Import all files in a directory tree

Question

I want to import all python files in a directory tree, i.e. if we have the following directory structure:

tests/
tests/foo.py
tests/subtests/bar.py

(Imagine that the tree is of arbitrary depth).

I would like to do import_all('tests') and load foo.py and bar.py. Importing with the usual modules names (tests.foo and tests.subtests.bar) would be nice, but is not required.

My actual use case is that I have a whole bunch of code containing django forms; I want to identify which forms use a particular field class. My plan for the above code is to load all of my code, and then examine all loaded classes to find form classes.

What's a nice, simple way to go about this in python 2.7?

Any reason not to use os.walk() http://www.tutorialspoint.com/python/os_walk.htm — Gary Walker, Mar 19 '14 at 21:05
@GaryWalker Not particularly, but os.walk is not a complete solution in itself. — Marcin, Mar 19 '14 at 21:08
[`__import__`](http://docs.python.org/3/library/functions.html#__import__) or better [`importlib`](http://docs.python.org/3/library/importlib.html#importlib.import_module) for Python 3.1+ — poke, Mar 19 '14 at 21:11
I guess I just assumed you would load modules dynamically. See http://stackoverflow.com/questions/769534/dynamic-loading-of-python-modules for an example. Not sure that this is necessarily a good idea -- depends up what you want to accomplish — Gary Walker, Mar 19 '14 at 21:12
@poke Unless I'm very much mistaken, `__import__` does not work with filenames, but module names. — Marcin, Mar 19 '14 at 21:14
@GaryWalker My purpose is explained in the question. As to that answer, it relies on `__import__`, which as far as I can tell does not use filenames, but module names. — Marcin, Mar 19 '14 at 21:15
@poke try doing `__import__('tests/subtests/bar.py')` and get back to me. — Marcin, Mar 19 '14 at 23:03
Well, *of course* that’s not how it works. But `__import__('tests.subtests.bar')` will work if `tests` and `tests.subtests` are packages. — poke, Mar 19 '14 at 23:10

score 0 · Accepted Answer · answered Mar 20 '14 at 16:04

Here's a rough and ready version using os.walk:

import os
prefix = 'tests/unit'
for dirpath, dirnames, filenames in os.walk(prefix):
    trimmedmods = [f[:f.find('.py')] for f in filenames if not f.startswith('__') and f.find('.py') > 0]
    for m in trimmedmods: 
        mod = dirpath.replace('/','.')+'.'+m
        print mod
        __import__(mod)

Gael Goret · Answer 2 · 2014-03-20T16:38:17.273

0

import os

my_dir = '/whatever/directory/'
files = [os.path.join(dirpath, f) for dirpath, dirnames, files in os.walk(my_dir) for f in files if f.endswith('.py')]
modules = [__import__(os.path.splitext(f)[0],globals(),locals(),[],-1) for f in files]

edited Mar 20 '14 at 16:38

answered Mar 20 '14 at 16:21

Gael Goret

62
7

Unless I'm mistaken this won't import anything in subdirectories. – Marcin Mar 20 '14 at 16:26
no for sure, the os.walk could solve the tree issue like in your example, but what I wanted to show here is the usage of import which allow to play with fully qualified name ... – Gael Goret Mar 20 '14 at 16:33

Python - Import all files in a directory tree

2 Answers2