Python - ModuleNotFoundError: No module named

Question

I'm new in Python and I'm having the following error with this simple example:

This is my project structure:

python_project
.
├── lib
│   ├── __init__.py
│   └── my_custom_lib.py
└── src
    ├── __init__.py
    └── main.py

And this is the error when I execute the src/main.py file:

☁  python_project  python src/main.py

Traceback (most recent call last):
  File "src/main.py", line 3, in <module>
    from lib import my_custom_lib
ImportError: No module named lib

If I move the main.py file to the root and then I execute this file again, it works... But it is not working inside src/ directory.

This is my main.py:

from lib import my_custom_lib

def do_something(message):
        my_custom_lib.show(message)

do_something('Hello World!')

Note: When I execute the same code from Pycharm, it is working fine, but not from my terminal.

Answer 1

Your PYTHONPATH is set to the parent directory of the executed script. So if the executed script is inside a directory src, it will never be able to find the sister directory lib because it isn't within the path. There's a few choices;

• Just move lib/ into src/ if it belongs to your code. If it's an external package, it should be pip installed.
• Have a top-level script outside of src/ that imports and runs src.main. This will add the top-level directory to python path.
• In src/main.py modify sys.path to include the top-level directory. This is usually frowned upon.
• Invoke src/main.py as a module with python -m src.main which will add the top-level directory to the python path. Kind of annoying to type, plus you'll need to change all your imports.

Answer 2

If I may add to MarkM's answer, if you wanted to keep your current directory structure and still make it work, you could add a setup.py in your root dir, where you can use setuptools to create a package you could install.

If your file had something along the lines of:

# setup.py

from setuptools import find_packages, setup

setup(
  name='foo',
  version=`1.0.0`,
  packages=find_packages(),
  entrypoints={
    'console_scripts': [
      'foo=src.main:main',
    ],
  },
)

And then you do pip install [--user] -e path/to/directory you'll get an "editable package" which will effectively a symlink to the package in your development directory, so any changes you make will not require a reinstall (unless of course you rejig package structure or add/remove/edit entry points).

This does assume your src/main.py has a main function.

You'll also need __init__.py files in your "package" directories, even in Python3, as otherwise Python assumes these are namespace packages (Won't go into detail) and the find_packages() call won't find them.

This will also allow your relative imports to work. Absolute imports will only work when invoking the script from your entry point but not when calling the script directly in your development directory.

Answer 3

You should have your main.py script above all python packages in your directory structure. Try to update your project to the following structure:

.
|__ main.py
|__ lib
|   |__ __init__.py
|   |__ your_custom_lib.py
|__ another_python_package
    |__ __init__.py
    |__ another_python_scripts

After that, python main.py in your project directory will work.

Answer 4

You are using the from a import b incorrectly. It should look like this:

import lib.my_custom_lib

The other method is used to import certain methods, functions, and classes from a module, not the module itself. To import a specific function from the my_custom_lib module, it would look like this:

from lib.my_custom_lib import foo

Answer 5

Try using a relative import instead:

from ..lib import my_custom_lib

Answer 6

in my case in visual code actual error in line 1

I didn't imported _typeshed module but by default it was their so delete that module if you found in line 1

Answer 7

MarkM's answer is still excellent; I'm using PyPi now. I attempted to disambiguate what he was saying about "current directory". In case my confusion was an intended feature, here is how I did it.

vagrant@testrunner:~/pypath$ tree
.
├── proga
│   └── script1.py
└── progb
    └── script1.py

script1.py is the same in both directories:

#!/usr/bin/env python3
import sys
print(sys.path)

Where I run it from makes no difference, PYTHONPATH prepends the directory containing the script I specify:

vagrant@testrunner:~/pypath/proga$ ./script1.py 
['/home/vagrant/pypath/proga', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/vagrant/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages']
vagrant@testrunner:~/pypath/proga$ ../progb/script1.py 
['/home/vagrant/pypath/progb', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/vagrant/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages']

Answer 8

For me importing with explicit paths works best in such situations. If you need to go up in the tree use '..'. The plumbing is a bit cumbersome, but it always works.

path = os.path.abspath(os.path.join(pathlib.Path(__file__).parent.absolute(), '..', 'subdir', 'myFile.py'))
loader = importlib.machinery.SourceFileLoader('myFile', path)
spec = importlib.util.spec_from_loader('myFile', loader)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)

# now use the module:
module.myMethod()
myClassInstance = module.myClass()