Import all files in current directory

Question

I have just started a python project. The directory structure is as follows:

/algorithms  
----/__init__.py  
----/linkedlist  
--------/__init__.py  
--------/file1.py  
--------/file2.py  
/tests  
----/test_linkedlist

You can also check the Github repository.

In each of the sub folders under algorithms, in the __init__ file I am including the following for all the files one by one:

from .file1 import *
from .file2 import *

And so on.

The task that I am trying to achieve is running all tests together using the query:

python3 -m unittest discover tests

Each file in the tests directory starts as follows:

from algorithms.linkedlist import *  
import unittest

Right now if I want to add a new file to the linkedlist directory, I create the file and then add another from .filename import * in the __init__ file.

How do I write a script in the __init__ file so that each time I create a new file, I do not have to manually insert the import command?

Roelant · Accepted Answer · 2019-03-07T12:15:27.417

1

So the __init__ is in the same folder? As the docs say The import statement is syntactic sugar for the __import__ function.

So we can use:

import importlib
import glob
for file in glob.iglob('*.py'):
    importlib.__import__(file)

Some reasons why this does not work:

You want to load the functions in the module - the import * from syntax. With this code you can only run file1.test.
You run the script loading from another directory, which confuses glob. We have to specify the actual working directory.
__import__ prefers to know the module name.

To find the solution I combine the import * from function from this answer with pkgutil.walk_packages from this blog.

import importlib
import pkgutil 

def custom_import_all(module_name):
    """ Use to dynamically execute from module_name import * """
    # get a handle on the module
    mdl = importlib.import_module(module_name)

    # is there an __all__?  if so respect it
    if "__all__" in mdl.__dict__:
        names = mdl.__dict__["__all__"]
    else:
        # otherwise we import all names that don't begin with _
        names = [x for x in mdl.__dict__ if not x.startswith("_")]

    # now drag them in
    globals().update({k: getattr(mdl, k) for k in names})


__path__ = pkgutil.extend_path(__path__, __name__)
for importer, modname, ispkg in pkgutil.walk_packages(path=__path__, prefix=__name__+'.'):
    custom_import_all(modname)

edited Mar 07 '19 at 12:15

answered Mar 07 '19 at 10:46

Roelant

4,508
1
32
62

Copy pasting this in the __init__ file is failing. Yes the init file is in the same folder. I am trying to run the test command given in the question from the root directory of the package. – Omar S Mar 07 '19 at 10:54
@OmarShalla What do you mean by failing? Do you mean that the files do not get included correctly and therefor your tests are failing or is the suggested code producing errors itself? – FlyingTeller Mar 07 '19 at 11:41
The suggested code is producing errors. You can clone my repository and try it yourself. The error log is too long to post here. Again I am running the following command 'python3 -m unittest discover tests' from the root directory of the package(the directory that contains the algorithms and tests folders) – Omar S Mar 07 '19 at 11:52
Testing with your git now :) – Roelant Mar 07 '19 at 11:58
Thanks for the help! – Omar S Mar 07 '19 at 12:01
Updated the answer. Just passed your 21 tests ;) – Roelant Mar 07 '19 at 12:15

Import all files in current directory

1 Answers1