Python unexpected import locations

Question

Small context: I'm writing a project in Python that uses Celery. I use a general framework of utility functions(that I wrote myself) that gets pip installed as a local module, the utility library is also used in other projects.

I have a project directory like this:

project/
  __init__.py
  celery.py
  test_runner.py
  some_more_files_that_dont_matter_for_this_example.py

And a utility project that I install as a python module that's like this:

__init__.py
utils/
  __init__.py
  utils.py
setup.py

Now, I have some code to initialize celery in the celery.py, this includes from utils.utils import Utils. The test_runner.py looks like this:

from utils.utils import Utils
if __name__ == '__main__':
    print 1+1

And my utils.py has the following import in it: from celery import Celery which SHOULD refer to the celery module I installed with pip install celery.

Now what goes wrong when I do python test_runner.py is that it tries to import utils.py, which then apperantly imports project/celery.py(not the celery module I pip-installed) and that makes a circular dependency happen(since project/celery.py imports utils/utils.py as well).

Can someone explain to me how to make sure the from celery import Celery in utils/utils.py only imports the actualy pip-installed celery module, and more importantly: why the imports happen like this? I'm running Python 3.5 and I thought that it would only do these local imports if I used from . import x.

Answer 1

The python interpreter adds the script's directory to sys.path (sort of reference). Furthermore, it is put at the beginning of the path and therefore searched first. This is why the local celery.py overrides the installed module.

You could either change sys.path and remove the local directory or put it at the back, but I think the better solution is not to name your local modules after global ones.

The from . import xyz syntax applies only to importing subpackages within packages (docs)