Python no module named X - absolute imports

Question

Before I even start I want to say - I know, that there are like bazillion questions similar to this, but I couldn't find the answer to my problem. I have a dir structure like this:

.
├── project
│   ├── A
│   │   ├── __init__.py
│   │   └── somelib.py
│   ├── B
│   ├── C
│   │   └── C
│   │       ├── foo.py
│   │       └── __init__.py
│   └── __init__.py
└── run.sh

run.sh:

python3 project/C/C/foo.py

foo.py:

from project.A.somelib import somefunc

---

VS Code actually gets the intellisense in foo.py - it tells me what funcitons/variables I can import from somelib. But when I run run.sh, I get this error message:

from project.A.somelib import somefunc
ModuleNotFoundError: No module named 'project'

Is there a way to solve this while preserving this directory structure?

---

• adding project/__init__.py changed nothing
• the sys.path in foo.py looks like this:

['/home/dabljues/projects/project/project/C/C', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/lib/python3.7/site-packages']

restrictions:

• I can't modify neither sys.path in the files nor PYTHONPATH before running the scripts
• I can't pip-install anything
• I don't have sudo-access
• I can't create a virtualenv, since the scripts are supposed to be downloadable and quickly executable

Answer 1

IDEs like VSCode or Pycharm make their own assumptions about a project, and will usually correctly link modules even if the interpreter that will ultimately run the code can't.

The reason why project.A.somelib can't be found is visible in your sys.path output, which gives you the places where python will search for modules. Since '/home/dabljues/projects/project/project' is not included, there is no way for python to resolve it during runtime.

---

A quick hack

You can just add the path manually to sys.path, either in the source file by running import sys; sys.insert(0, '/home/dabljues/projects/project/project/') in foo.py before any other imports happen, or by running export PYTHONPATH="${PYTHONPATH}:/home/dabljues/projects/project/project/" in your shell before run.sh.

---

Installing the project

Since it looks like you're developing a library, you might as well use the mechanisms python offers to make libraries shareable and thereby fixing any import issues. Add a minimal setup.py to the project root (i.e. /home/dabljues/projects/project/project/setup.py):

from setuptools import setup, find_packages


setup(
    name='project',
    version='0.1.0',
    packages=find_packages('project'),
)

And install your project in editable mode:

$ python3 -m pip install -e .

This will put a link in your python3 executable's site-packages that points to the project root, which makes it accessible whenever you run anything with python3.

---

Tests

I included print(__name__) at the top of all python files to get some output.

running run.sh without installing the package:

$ sh run.sh 
Traceback (most recent call last):
  File "project/C/C/foo.py", line 1, in <module>
    from project.A.somelib import somefunc
ModuleNotFoundError: No module named 'project'

after installing it

$ sh run.sh 
__main__
project.A.somelib

As you can see, project.C.C.foo is executed as a script, yet it finds all imports that start with project because project is installed.

Answer 2

Run python in package mode helps.

1) Add __init__.py for every path:

.
├── project
│   ├── A
│   │   ├── __init__.py
│   │   └── somelib.py
│   ├── B
│   ├── C
│   │   ├── __init__.py
│   │   └── C
│   │       ├── foo.py
│   │       └── __init__.py
│   └── __init__.py
└── run.sh

2) Import module with relative path in foo.py:

from ...A.somelib import somefunc

3) Run python in package mode:

python -m project.C.C.foo

It works for me.

Answer 3

I'm unable to reproduce this given your code (assuming run.sh just starts a script).

Are you sure it's not a case of e.g. circular imports?

$ mkdir -p project/A project/C/C
$ cat > project/C/C/foo.py
print('moof!')
$ cat > project/A/somelib.py
print('beef!')
$ cat > script.py
import project.A.somelib
import project.C.C.foo
$ tree
.
├── project
│   ├── A
│   │   └── somelib.py
│   └── C
│       └── C
│           └── foo.py
└── script.py
$ python3 script.py
beef!
moof!

Answer 4

Change your run.sh script to run foo as a module. python3 -m proj.C.C.foo