Python imports that work both when using a file as a module and when called directly

Question

I was reading today about absolute and relative imports in Python, and I was wondering if there is a pythonic way of making an import work when the script that contains the import is used as a module and when it is called directly. As an example:

└── project
      ├── main.py
      └── package
             ├── submoduleA.py
             └── submoduleB.py

main.py

from package.submoduleA import submoduleAfunction
submoduleAfunction()

submoduleB.py

def submoduleBfunction():
    print('Hi from submoduleBfunction')

submoduleA.py

# Absolute import, works when called directly: python3 package/submoduleA.py 
from submoduleB import submoduleBfunction 

# Relative import, works when imported as module: python3 main.py
# from .submoduleB import submoduleBfunction 

def submoduleAfunction():
    submoduleBfunction()
    
if __name__=="__main__":
    submoduleAfunction()

As you see, I have there both absolute and relative imports. Each of them works in a specific situation. But is there a pythonic way of making it work for both cases? So far I've done the trick messing with sys.path, but I'm not sure that this will work in every situation (i.e. called from interpreter or in different OS's) or even if this goes against any recommendations.

import sys
import pathlib
sys.path.append(str(pathlib.Path(__file__).parent.resolve()))

At work we have a root folder on the path, and everything is in subdirectories inside it, so everything imports from the top level. For example, if you run `company/scripts/something.py`, then that might have `from company.scripts.something_else import x` — Peter, Jun 30 '21 at 19:14

JL Peyret · Answer 1 · 2021-07-01T02:43:21.163

1

Disclaimer:

Your mileage may vary and I realize that my answer does not fully answer the question (using relative imports). Mea maxima culpa (my bad).
In my defense: googling python avoid relative imports returns 540k hits.

I usually avoid relative imports because they seem to have too many special cases on just this type of things and I am a huge fan of running python files directly as scripts when appropriate.

For example, my constants.py file can be run as a script - it will breakpoint() and I can inspect its contents, a good deal of which come from environment variables.

Assuming that the parent directory for project is in your Python path, I would just use from project.package.submoduleA import submoduleAfunction. That works in all cases.

(One possible additional benefit I found with explicit imports is when I wanted to rename my top-level package (myproject to myproject2). Having everything explicit allowed me to quickly sed the changes. Note Wim's remarks in the comments below)

edited Jul 01 '21 at 02:43

answered Jul 01 '21 at 01:34

JL Peyret

10,917
2
54
73

The last point doesn't really make sense to me. If you use relative imports, then there is no need to replace import statements at all during a rename (only changing the top-level package name is needed), so how is that is an "additional benefit" of absolute imports? – wim Jul 01 '21 at 02:37
Don't really recall except that, when I did it, it did seem that being very explicit and importing from the top down made it easier. I did reorganize my code by using `pip install -E ` on the equivalent of the OP's parent dir however, so that might have been part of it. – JL Peyret Jul 01 '21 at 02:42
Oh, and one reason from my *anti-patterning* is that I write celery tasks that run massive batches. Those are fully exercised and debugged from the command line, not via celery launches. Even end users can also run them from the CLI. Celery is very much relegated to being a barebones invocation method but it does need to import those batches. – JL Peyret Jul 01 '21 at 03:00

wim · Accepted Answer · 2021-07-01T01:57:12.330

# Absolute import, works when called directly: python3 package/submoduleA.py 
from submoduleB import submoduleBfunction

Since submoduleA is the "sibling" of submoduleB, this is actually an implicit relative import. The style guide says of them:

Implicit relative imports should never be used and have been removed in Python 3.

The only reason it even works when running python3 package/submoduleA.py as a script is that Python prepends the directory of the script to sys.path. From the docs:

As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter.

That is, the directory /path/to/package gets injected to sys.path, allowing submoduleB to be imported directly.

Implicit relative imports should never be used, so what should you use instead? It doesn't matter, you can use the correct form of the absolute import:

from package.submoduleB import submoduleBfunction

or a relative import:

from .submoduleB import submoduleBfunction

Either is fine as long as package has been installed into the environment (installing the package is what will put the code into site-packages, which is on sys.path).

If you absolute must run package/submoduleA.py directly as a script, which is considered an anti-pattern, then you will need to use the absolute import version. Relative imports do not work in this case. You would need to use instead python3 -m package.submoduleA in order for the relative-import version to work.

score -1 · Answer 3 · answered Jun 30 '21 at 20:15

-1

One possibility is to just enclose the absolute import in a try block and catch an ImportException. In case of an exception, you are presumably using it as a module so then do a relative import.

try:
    from submoduleB import submoduleBfunction 
except ImportException:
    from .submoduleB import submoduleBfunction

answered Jun 30 '21 at 20:15

Tyberius

625
2
12
20

This is not safe, in the general case `mod` and `.mod` may refer to entirely different packages. – wim Jul 01 '21 at 02:34

Python imports that work both when using a file as a module and when called directly

3 Answers3

Disclaimer: