Do Python module imports depend on how the code is run?

Question

I don't think any language causes as much headache as python in so simple a thing as importing other source files. So here is the question: Do my module imports need to depend on how the code is supposed to run?

I have the following directory structure:

./__init__.py
./config.py
./kmer
./kmer/__init__.py
./kmer/__main__.py
./kmer/__pycache__
./kmer/__pycache__/__init__.cpython-36.pyc
./kmer/__pycache__/__main__.cpython-36.pyc
./kmer/__pycache__/bed.cpython-36.pyc
./kmer/__pycache__/config.cpython-36.pyc
./kmer/__pycache__/reference.cpython-36.pyc
./kmer/__pycache__/sets.cpython-36.pyc
./kmer/bed.py
./kmer/config.py
./kmer/reference.py
./kmer/sets.py

I wish to import a module inside the khmer from another module inside the kmer package. Simple?

So I add this at the of the bed.py:

import reference
import config
import sets

Now thing will work just fine if I run python bed.py from the kmer directory. Also things are fine if I come back one directory and call python kmer/bed.py. Seems like python searches for imported modules relative to the given file.

Again running it as python -m bed from the kmer directory works fine but coming back one directory and running python -m kmer.bed results in module errors. Here it seem like python looks for modules inside the interpreter's current directory which could be anywhere on the file system so imports relative to it shouldn't work.

This basically means that the imports should depend on how the code is going to be run which makes no sense. I'd appreciate some explanation on how this is supposed to work.

I've looked at quite a lot of resources including the answer to this question which although very detailed (one of the best descriptions I've found actually) doesn't solve my problem and also doesn't provide proper examples.

Update: I believe this question is more focused on a different aspect of relative import, more precisely how different methods of running the code affect the imports, than the one it is marked as a duplicate of. Thats why I mentioned the other question in first place. Hence I don't think this should be a duplicate.

Answer 1

When writing a package, you must consistently treat it as a package. First, that means using relative imports within the package: write from . import reference in bed.py.

It also means the answer for how to access it is always the same: put the directory containing kmer onto sys.path (usually via PYTHONPATH). This is true even if running some module in the package as the script, so it must be python -m kmer.bed.

Unfortunately, it's a bad idea to run a module inside a package as a script: the module's file is still eligible to be imported again under its proper name (rather than the __main__ used the first time). The only robust solution is to write a __main__.py and run the package as the script, avoiding overloading a public module name.

Finally, don't pay much attention to the "feature" of Python putting the script's directory on sys.path. This is useless other than for toy problems where you don't care about name collisions, since any script whose directory would be useful on sys.path is necessarily a module (except in case of tricks like making its name not be an identifier). Moreover, if you want your library to be available to scripts installed elsewhere, it needs to get on sys.path some other way anyway.