0

(TL;DR) I am creating a Python package and I always have trouble with importing and understanding how python runs scripts within packages/modules. How can I correctly import between directories the following directory structure? Where should I place the script that I would like to be the entry point for this application, and how should the user run it?

My current directory structure looks like this:

package_name
├── (venv)
├── __init__.py
├── package_name
│   ├── __init__.py
│   ├── script.py
│   ├── common
│   │   ├── __init__.py
│   │   ├── cfg1.py
│   │   └── cfg2.py
│   ├── core
│   │   ├── __init__.py
│   │   ├── utils1.py
│   │   └── utils2.py
│   └── templates
│       ├── base.txt
│       ├── template1.txt
│       ├── template2.txt
│       ├── ...
│       ├── templateN.txt
├── setup.py
└── tests
    └── test_core
        ├── data
        └── test_core.py

Inside script.py:

from core import utils1, utils2

def main():
    # Parse some args - want the user to supply them from command line
    utils1.thing()
    utils2.another_thing()

if __name__ == "__main__":
    main()

Inside utils1.py:

from package_name.common.cfg1 import allowed_configs
# This fails
# And then the utilities are here

I want the user to be able to call script.py and supply command line arguments to do certain things. When I run:

python3 pathto/package_name/package_name/script.py option1 --arg1 value

from the command line after activating my virtualenv, I get an error from utils1.py:

ModuleNotFoundError: No module named 'package_name.common'

If I run it as a module:

python3 -m package_name.package_name.script option1 --arg1 value

I get a different error on this line:

from core import utils1, utils2

The error:

ModuleNotFoundError: No module named 'core'

Now from my research when it comes to modules/packages/scripts, I understand I can't run a module within a package as a script without using the -m flag. I think it makes sense that I'd get that error, but I don't intuitively understand why.

I'd really like to not have to mess with the Python path manually, but I could use some advice on the following:

1) What the correct structure/layout for a Python package like this that is intended to be distributed to users and called from the command line? How should the end user eventually invoke script.py?

2) How do I import between subdirectories without manually setting my path in the script? Is it possible?

3) I haven't "installed" the package with setup.py yet, since I still haven't figured out exactly how that works. Will fixing up my setup.py to have packages=find_packages() solve this issue? If so, do I need to run setup.py development every time I want to test a change to my code?

bgbp
  • 3
  • 2

1 Answers1

0

I was able to get it to work in the following way:

I wanted to be able to run a script from the command line, so I followed along with this page in order to understand how to implement an entry point in my setup.py: https://click.palletsprojects.com/en/5.x/setuptools/

I tried to add a scripts keyword but I couldn't quite figure out how to make it work. By using console_scripts I was able to call my script directly like an executable after I installed it with pip install -e . . For others who have the same issue, note that when I used pip to install this, I did it from the directory that held setup.py (in hindsight that makes sense, but I tried to install the first time from above the top package_name directory)

The following changed in my imports. Originally, I had from core import utils1, utils2 but when I installed the package with pip, it changed to package_name.core (since package_name is the package). The same goes for the other failing import path that had an error; the correct import is package_name.common.cfg1

Hope this helps someone! References I found helpful:

bgbp
  • 3
  • 2