Python namespace sub-package depending on another sub-package fails to install

Question

My goal is to create a namespace package with two sub-packages: foo and bar (dependent on foo), and push the namespace package to a git repo (without publishing it to PyPI) so that I can install either sub-package with the repo url.

I'm following this tutorial to create the namespace structure:

root/
 ├ setup.py
 └ microlibs/
    ├ foo/
    │  ├ setup.py
    │  ├ macrolib/
    │     └ foo/
    │        ├ __init__.py
    │        ├ module1.py
    │        ├ ...
    │        └ moduleN.py
    .
    .
    .
    └ bar/
       ├ setup.py
       ├ macrolib/
          └ bar/
             ├ __init__.py
             ├ module1.py
             ├ ...
             └ moduleN.py

The setup.py file of foo has no dependencies:

foo/setup.py

from setuptools import setup
microlib_name = 'macrolib.foo'
setup(
    name=microlib_name,
    version="0.1.0",
    namespace_packages=['macrolib'],
    packages=[microlib_name],
    install_requires=[]
)

Since bar's dependencies include foo, the setup.py of bar includes macrolib.bar in install_requires list:

bar/setup.py

from setuptools import setup
microlib_name = 'macrolib.bar'
setup(
    name=microlib_name,
    version="0.1.0",
    namespace_packages=['macrolib'],
    packages=[microlib_name],
    install_requires=[
        'macrolib.foo'
    ]
)

After pushed to bitbucket repo, I can install macrolib.foo without problems with a subdirectory of the repo.

$ pip install git+http://path/to/repo.git@<branch name>#"subdirectory=foo&egg=macrolib.foo"

With macrolib.foo installed, I can also install macrolib.bar without problems with a subdirectory of the repo.

$ pip install git+http://path/to/repo.git@<branch name>#"subdirectory=bar&egg=macrolib.bar"

However, if I try to install macrolib.bar without installing macrolib.foo first, the installation failed.

$ pip install git+http://path/to/repo.git@<branch name>#"subdirectory=bar&egg=macrolib.bar"

error:

Collecting macrolib.foo (from macrolib.bar==0.1.0)
  Could not find a version that satisfies the requirement macrolib.foo (from macrolib.bar==0.1.0) (from versions: )
No matching distribution found for macrolib.foo (from macrolib.bar==0.1.0)

I'm guessing this is because of the missing of dependency_links in bar/setup.py. So I tried different combinations of link urls, all failed with the same error.

Formats I have tried:

dependency_links=['http://path/to/repo.git@<branch name>#"subdirectory=foo&egg=macrolib.foo"']

dependency_links=['http://path/to/repo.git@<branch name>#subdirectory=foo&egg=macrolib.foo']

dependency_links=['http://path/to/repo/tarball/<branch name>#"subdirectory=foo&egg=macrolib.foo"']

dependency_links=['http://path/to/repo/tarball/<branch name>#subdirectory=foo&egg=macrolib.foo']

dependency_links=['http://path/to/repo/archive/<branch name>.zip#"subdirectory=foo&egg=macrolib.foo"']

dependency_links=['http://path/to/repo/archive/<branch name>.zip#subdirectory=foo&egg=macrolib.foo']

OR add a prefix 'git+' to all the above urls.

My question is what is the correct url format for dependency_links in order to install macrolib.foo as a dependency, or is there any other ways to make it work?

score 2 · Accepted Answer · edited Jun 15 '18 at 19:41

2

This is the correct format (adding 'git+' and dependency version):

dependency_links=['git+http://path/to/repo.git@<branch name>#subdirectory=foo&egg=macrolib.foo-0.1.0']

and you need to ask pip to process it:

pip install --process-dependency-links git+http://path/to/repo.git@<branch name>#"subdirectory=bar&egg=macrolib.bar"

edited Jun 15 '18 at 19:41

Chenlu

449
1
6
19

answered Jun 15 '18 at 15:43

phd

82,685
13
120
165

1

Thanks. Adding `--process-dependency-links` indeed makes one step closer to the solution, but it still causes the same error due to the unrecognized link format. I just figured out that adding `git+` and 'the dependency version' makes it correct. I edited the answer a little bit so that it can be accepted. – Chenlu Jun 15 '18 at 17:23
I still have a question. I can see many others using tarball/ as the dependency_link e.g. [this one](https://stackoverflow.com/a/26062864/6538935). Why does the .git link also work? – Chenlu Jun 15 '18 at 17:33
1

Because `pip` can run `svn/hg/git clone` and run `python setup.py install` in the cloned worktree. – phd Jun 15 '18 at 20:41

Python namespace sub-package depending on another sub-package fails to install

1 Answers1

Linked