Python imports - ModuleNotFoundError: No module named X

Question

I have read probably all of the posts on here regarding imports and I still cannot figure out what is going on with the imports, I have spent hours trying to get a very simple example working and am literally pulling my hair out.

I am using python 3.7 and pycharm but I am running my code from the commandline, for the unit tests I am using pytest.

My project structure is:

my_message_validator/
    __init__.py

    module_1/
        __init.py__
        foo.py

    module_2/
        __init.py__
        bar.py
        baz.py

    module_3
        context.py
        test_all.py

module_1.init.py

from module_1 import foo

module_2.init.py

# For some reason pycharm doesnt complain when I use '.' but if I use module_2 it does
from . import bar, baz

If I try to run my code or my tests from the commandline no matter how I move things around I seem to get either ModuleNotFoundError: No module named, when I have managed to get the tests working I still cannot run my code on its own from the commandline.

How can I import module_1 into module_2 and are these actually packages? I am coming from java and find the imports a lot easier to understand, I am finding the python importing very confusing...

Also how can I can then import whatever I need into my test module\package\folders context.py?

Currently the test context look like:

import os
import sys

# Is this needed as it doesnt seem to do anything?
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

from module_1.foo import Foo

from module_2 import bar, baz

In test_all.py I am trying to import from the context file like this:

from .context import bar,baz
from .context import Foo

# Calling in test like
Foo.load_file(file)

bar.method_one()
baz.method_two()

Do I need all the __init.py__ files and what should I be putting in them to make my methods and classes public\exposed? I would like this entire package to be reusable so want to be able to treat it like a jar file in java.

Any help would be much appreciated as it seems everytime I change something I get an error in a different place, python seems so much more complicated than java right now.

Have you tried importing like: from my_message_validator.module_1.foo import Foo ? — Pratibha Gupta, Nov 14 '19 at 23:59

Pierre · Accepted Answer · 2021-04-13T17:13:35.987

8

First, do not use relative imports (with .), as it is known for causing multiple issues. Always write your imports relative to the root of your project. For example, you did it well for from module_1.foo import Foo. You should also do it in test_all.py and context.py. Moreover, after using relative imports, the __init__.py files can be left empty in your case.

Most likely, the Python interpreter cannot find your modules because the PYTHONPATH environment variable does not contain the root of your project. If you run export PYTHONPATH="YOUR_PROJECT_ROOT_ABSOLUTE_PATH:$PYTHONPATH" before your script, it should run as expected. To make sure this variable is set all the time, you can add the export statement to your shell profile file (e.g. .bashrc or .bash_profile).

After chatting with the author, it turns out there was a fourth issue. It was a name collision like the one in this other question. In his project directory, module_1 was actually called foo like its child foo.py, which confused the interpreter.

edited Apr 13 '21 at 17:13

answered Nov 15 '19 at 00:10

Pierre

1,068
1
9
13

So would you suggest I get rid of all of the `__init__.py files`? If bar and baz only contain functions and not classes what is the correct way to import them? – berimbolo Nov 15 '19 at 00:14
Yes indeed, I suggest you delete all `__init__.py` files. Regarding `bar` and `baz`, you can import functions from them similarly to how you import classes in Java. That is, you can do `from module_2.bar import my_bar_function` and `from module_2.baz import my_baz_function`. Then, you can simply call `my_bar_function()` and `my_baz_function()` in your script. – Pierre Nov 15 '19 at 00:20
And is there any point in using a `context.py` for the tests? I followed a tutorial on how to create packages and it seems to be handling the imports for test suites which then import from `.context` – berimbolo Nov 15 '19 at 00:25
It generally depends on how much code is shared between your different tests. I would suggest you start the project without `context.py`, and then add it later in case you find yourself importing the same things all the time across your different tests. – Pierre Nov 15 '19 at 00:30
Ok thanks for the help, I have trimmed everything down so that I have no `__init.py__` files and no test context, unfortunately I cannot even import `baz` in `bar` now and my tests fail with `ModuleNotFoundError: No module named 'module_1'`, I exported the `PYTHONPATH` as you suggested but none of the suggestions has worked. – berimbolo Nov 15 '19 at 00:35
`(my_message_validator) validator $ echo $PYTHONPATH /Users/berimbolo/PycharmProjects/my_message_validator:` – berimbolo Nov 15 '19 at 00:39
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/202369/discussion-between-pierre-and-berimbolo). – Pierre Nov 15 '19 at 00:39
From [PEP 420 - Implicit Namespace Packages](https://www.python.org/dev/peps/pep-0420/#specification): "Regular packages will continue to have an `__init__.py` and will reside in a single directory." Also https://stackoverflow.com/a/48804718. – djvg Apr 13 '21 at 07:05
1

@djvg Thanks for the comment. I agree, and would add that some linters or static analysis tools won't explore directories that are not regular packages. I edited the answer. – Pierre Apr 13 '21 at 17:15

score 0 · Answer 2 · answered Nov 15 '19 at 00:00

0

Have you tried importing like:

from my_message_validator.module_1.foo import Foo
from my_message_validator.module_2 import bar, baz

answered Nov 15 '19 at 00:00

Pratibha Gupta

199
10

I had thought that using the `__init__.py` files and then exporting there (using import) would mean I wouldnt have to give the full path to everything. What is the norm when developing with python? – berimbolo Nov 15 '19 at 00:02
If I would use the absolute path over relative paths do I need to bother with `__init__.py` files at all? – berimbolo Nov 15 '19 at 00:03
When I have added the absolute path to my unit test then I get this error instead `ModuleNotFoundError: No module named 'my_message_validator'` although pycharm seems to be happy with this format – berimbolo Nov 15 '19 at 00:06
absolute path for import is required as Python is recognizing your project path by root directory. Rest of the modules you'll require to mention root onwards. – Pratibha Gupta Nov 15 '19 at 00:07
`Rest of the modules you'll require to mention root onwards`, could you update your answer with an example please, I dont know what you mean. – berimbolo Nov 15 '19 at 00:08
sorry for confusing answer, If my_message_validator is your root path for the project, you need to start importing your modules from my_message_validator. python doesn't recognize your internal modules unless other module is also in the same directory. – Pratibha Gupta Nov 15 '19 at 00:14
I have given the absolute path and I get this error `ImportError: attempted relative import with no known parent package`, is there any point in using a relative path or the test context file at all? – berimbolo Nov 15 '19 at 00:18

score 0 · Answer 3 · edited Jan 21 '20 at 09:16

0

I had the same case.
I started the application in this way:

flask run

And every time I got a ModuleNotFoundError error on the website.
When I started the application like this:

python3 -m flask run

the application started without errors :-) .

edited Jan 21 '20 at 09:16

EstevaoLuis

2,422
7
33
40

answered Jan 21 '20 at 08:06

Tomasz

1
2

Python imports - ModuleNotFoundError: No module named X

3 Answers3