Pip install sub-package from own (namespace) package in python

Question

I'd like to install some special sub-package from a package.

For example, I want to create package with pkg_a and pkg_b. But I want to allow the user to choose which he wants to install.

What I'd like to do:

git clone https://github.com/pypa/sample-namespace-packages.git
cd sample-namespace-packages
touch setup.py

setup-py:

import setuptools

setup(
    name='native',
    version='1',
    packages=setuptools.find_packages()
)

# for all packages
pip install -e native #Successfully installed native

# for specific
# Throws ERROR: native.pkg_a is not a valid editable requirement. 
# It should either be a path to a local project
pip install -e native.pkg_a native.pkg_b

# for specific
cd native
pip install -e pkg_a # Successfully installed example-pkg-a

I've seen this in another questions answer so it must be possible: Python install sub-package from package

Also I've read the Packaging namespace packages documentation and tried to do the trick with the repo

I've tried some variants with an additional setup.py in the native directory, but I can't handle it and I am thankful for all help.

Update

In addition to the answer from sinoroc I've made an own repo. I created a package Nmspc, with subpackages NmspcPing and NmspcPong. But I want to allow the user to choose which he wants to install. Also it must to be possible to install the whole package.

What I'd like to do is something like that:

git clone https://github.com/cj-prog/Nmspc.git
cd Nmspc

# for all packages
pip install Nmspc

# Test import
python3 -c "import nmspc; import nmspc.pong"

# for specific
pip install -e Nmspc.pong # or 
pip install -e pong

# Test import
python3 -c "import pong;"

Answer 1

A solution for your use case seems to be similar to the one I gave here: https://stackoverflow.com/a/58024830/11138259, as well as the one you linked in your question: Python install sub-package from package.

Here is an example...

The directory tree might look like this:

.
├── Nmspc
│   ├── nmspc
│   │   └── _nmspc
│   │       └── __init__.py
│   └── setup.py
├── NmspcPing
│   ├── nmspc
│   │   └── ping
│   │       └── __init__.py
│   └── setup.py
└── NmspcPong
    ├── nmspc
    │   └── pong
    │       └── __init__.py
    └── setup.py

3 Python projects:

• NmspcPing provides nmspc.ping
• NmspcPong provides nmspc.pong
• Nmspc depends on the other two projects (and also provides nmspc._nmspc see below for details)

They are all namespace packages. They are built using the instructions from the Python Packaging User Guide on "Packaging namespace packages, Native namespace packages". There is another example here.

The project Nmspc is basically empty, no actual code, but the important part is to add the other two NmspcPing and NmspcPong as installation requirements. Another thing to note, is that for convenience it is also a namespace package with nmspc._nmspc being kind of hidden (the leading underscore is the naming convention for hidden things in Python).

NmspcPing/setup.py (and similarly NmspcPong/setup.py):

#!/usr/bin/env python3

import setuptools

setuptools.setup(
    name='NmspcPing',
    version='1.2.3',
    packages=['nmspc.ping',],
)

Nmspc/setup.py:

#!/usr/bin/env python3

import setuptools

setuptools.setup(
    name='Nmspc',
    version='1.2.3',
    packages=['nmspc._nmspc',],
    install_requires=['NmspcPing', 'NmspcPong',],
)

Assuming you are in the root directory, you can install these for development like this:

$ python3 -m pip install -e NmspcPing
$ python3 -m pip install -e NmspcPong
$ python3 -m pip install -e Nmspc

And then you should be able to use them like this:

$ python3 -c "import nmspc.ping; import nmspc.pong; import nmspc._nmspc;"

---

Update

This can be simplified:

.
├── NmspcPing
│   ├── nmspc
│   │   └── ping
│   │       └── __init__.py
│   └── setup.py
├── NmspcPong
│   ├── nmspc
│   │   └── pong
│   │       └── __init__.py
│   └── setup.py
└── setup.py

setup.py

#!/usr/bin/env python3

import setuptools

setuptools.setup(
    name='Nmspc',
    version='1.2.3',
    install_requires=['NmspcPing', 'NmspcPong',],
)

Use it like this:

$ python3 -m pip install ./NmspcPing ./NmspcPong/ .
$ python3 -c "import nmspc.ping; import nmspc.pong;"

Answer 2

If the projects are not installed from an index such as PyPI, it is not possible to take advantage of the install_requires feature. Something like this could be done instead:

.
├── NmspcPing
│   ├── nmspc.ping
│   │   └── __init__.py
│   └── setup.py
├── NmspcPong
│   ├── nmspc.pong
│   │   └── __init__.py
│   └── setup.py
└── setup.py

NmspcPing/setup.py (and similarly NmspcPong/setup.py)

import setuptools

setuptools.setup(
    name='NmspcPing',
    version='1.2.3',
    package_dir={'nmspc.ping': 'nmspc.ping'},
    packages=['nmspc.ping'],
)

setup.py

import setuptools

setuptools.setup(
    name='Nmspc',
    version='1.2.3',
    package_dir={
        'nmspc.ping': 'NmspcPing/nmspc.ping',
        'nmspc.pong': 'NmspcPong/nmspc.pong',
    },
    packages=['nmspc.ping', 'nmspc.pong'],
)

This allows to install from the root folder in any of the following combinations:

$ python3 -m pip install .
$ python3 -m pip install ./NmspcPing
$ python3 -m pip install ./NmspcPong
$ python3 -m pip install ./NmspcPing ./NmspcPong