1

I have a question about importing modules inside a python package. I do not know how to do it such that I can run the python code, and also import the package somewhere else.

Setup

Here is the minimal example's folder structure

.
├── test_package/
│   ├── __init__.py
│   ├── module_A.py
│   └── module_B.py
│
├── LICENCE
├── README.md
└── setup.py

The files contain

_init_.py

from .module_A import *
from .module_B import *

module_A.py

from module_B import c

def print_variable():
    print(c)

if __name__ == "__main__":
    print_variable()

module_B.py

c = 5

setup.py

import setuptools

setuptools.setup(
    name="test_package",
    version="0.0.1",
    author="Example Author",
    author_email="author@example.com",
    description="A small example package",
    packages=setuptools.find_packages(),
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    python_requires='>=3.6',
)

The problem

If I simply run

python test_package/module_A.py

I get the expected output of 5. So far, so good.

Now I want to install the package (e.g. to run unit tests on it from a separate folder). I do this via

pip install .

from the root folder. The installation works fine, but if I now try to import the package, I find

python
Python 3.7.3 (default, Mar 27 2019, 16:54:48)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import test_package
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/package/test_package/__init__.py", line 1, in <module>
    from .module_A import *
  File "/path/to/package/test_package/test_package/module_A.py", line 1, in <module>
    from module_B import c
ModuleNotFoundError: No module named 'module_B'
>>>

When importing module_A, it seems to be unable to locate module_B.

What I tried to fix it

I can adjust the import line in module_A.py by adding a dot . to

from .module_B import c
# ...

Then I can import the package and use its function.

python
Python 3.7.3 (default, Mar 27 2019, 16:54:48)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import test_package
>>> test_package.print_variable()
5

However, I can no longer run the script.

python test_package/module_A.py
Traceback (most recent call last):
  File "test_package/module_A.py", line 1, in <module>
    from .module_B import c
ModuleNotFoundError: No module named '__main__.module_B'; '__main__' is not a package

Now the script can not find module_B anymore.

So, if I use a relative import and include the ., the package import works well, but the script no longer finds the module. On the other hand, without the ., I can run the script, but not import the package after installation. What am I doing wrong?

Bitswazsky
  • 4,242
  • 3
  • 29
  • 58
user157765
  • 85
  • 8
  • 1
    If the thing you are trying to run is a module that is part of a package (it has `__init__.py` and relative imports and all these things), then you need to run it `python -m test_package.module_A`. It can not be run as a simple script anymore. It's probably more complicated than that, but that's the short version. – sinoroc Jun 22 '20 at 14:26
  • Thanks for your reply. If I run the code this way, it works. However, I do get a RuntimeWarning: ```python3.7/runpy.py:125: RuntimeWarning: 'test_package.module_A' found in sys.modules after import of package 'test_package', but prior to execution of 'test_package.module_A'; this may result in unpredictable behaviour warn(RuntimeWarning(msg))``` – user157765 Jun 22 '20 at 14:30
  • Don't know, might be worth a different question. By chance, did you do any changes to `PYTHONPATH` or `sys.path`? Those are often unnecessary, and rather counter-productive, even though they are often recommended (for dubious or ill-informed reasons). -- Or maybe read this: https://stackoverflow.com/a/45070583/11138259 – sinoroc Jun 22 '20 at 14:34
  • Hmm, I saw that some people recommended this, and I experimented with this earlier today without success. Do you think I might have to revert my ```PYTHONPATH```? – user157765 Jun 22 '20 at 14:44
  • Does this answer your question? [Import a module from both within same package and from outside the package in Python 3](https://stackoverflow.com/questions/47319423/import-a-module-from-both-within-same-package-and-from-outside-the-package-in-py) – MisterMiyagi Jun 22 '20 at 14:53
  • @sinoroc Update: If I empty the ```__init__.py``` file, the warning goes away, but then I can no longer use the ```print_variable()``` function after import of the package. I am very confused. @MisterMiyagi It looks like my problem. I will try out the proposed solution. It really feels like I'm doing something wrong. This should not be that hard. – user157765 Jun 22 '20 at 15:02
  • 1
    Do not modify `PYTHONPATH` or `sys.path`. I still haven't understood why it is recommended so often. It always leads to issues, when the actual fixes are simple and robust. So yes, if you have modified them earlier, then remove these modifications. And follow @MisterMiyagi advice, it seems sound. If the issue persists, then create a new question with only the new relevant bits, not the parts that we know don't work. – sinoroc Jun 22 '20 at 15:44
  • 1
    Thanks a lot for the replies. I think the main reason for my confusion and the resulting warnings and errors are my attempt to mix python scripts into a package. It might simply be a bad idea, and it could be better to simply have a package without any scripts, and then external scripts importing that package. Then everything works fine and seems cleaner. I will write up an answer tomorrow to post here, if this makes sense. – user157765 Jun 22 '20 at 15:51
  • 1
    Note that if you want to have scripts belonging to a package, [``entry_points`` ``console_scripts``](https://packaging.python.org/specifications/entry-points/) might be preferable. – MisterMiyagi Jun 22 '20 at 16:25

0 Answers0