Installing dependency from a VCS repo subdirectory using setuptools

Question

I'm trying use setuptools to install a dependency from a VCS and inside a subdirectory.

My setup.py looks like this:

#!/usr/bin/env python3

from setuptools import setup

required = [
    "package"
]
dependency_links = [
    "git+ssh://git@host/repo.git@tag#subdirectory=subdir#egg=package-version"
]

setup(install_requires=required, dependency_links=dependency_links)

Running python3 setup.py install in a virtualenv, I get the following error:

Download error on git+ssh://git@host/repo.git@tag#subdirectory=subdir#egg=package-version: unknown url type: git+ssh -- Some packages may not be found!

For the sake of debugging, I used the following public Github repo instead:

required = [
    "pycocotools"
]
dependency_links = [
    "git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI#egg=pycocotools-2.0"
]

This example was suggested here as a solution to a similar question. I got the same unknown url type error (the package is eventually retrieved through PyPI, not through the VCS URL !).

What I've tried

git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI#egg=pycocotools-2.0

• python3 setup.py install: unknown url type: git+https -- Some packages may not be found!
• pip3 install: ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/tmp/pip-install-lwpbj7yv/pycocotools-2.0/PythonAPI#egg=pycocotools-2.0': '/tmp/pip-install-lwpbj7yv/pycocotools-2.0/PythonAPI#egg=pycocotools-2.0'

git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI&egg=pycocotools-2.0

• python3 setup.py install: unknown url type: git+https -- Some packages may not be found!
• pip3 install: OK, but WARNING: Generating metadata for package pycocotools-2.0 produced metadata for project name pycocotools. Fix your #egg=pycocotools-2.0 fragments.

git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI&egg=pycocotools

• python3 setup.py install: unknown url type: git+https -- Some packages may not be found!
• pip3 install: OK

I've also tried removing the git+ for all these URLs, but it cannot detect archive format.

Versions I'm using

• setuptools 46.4.0
• python 3.6.9
• pip 20.1.1

Answer 1

dependency_links were declared obsolete and finally removed in pip 19.0. The replacement for it is install_requires with special syntax (supported since pip 19.1):

install_requires=[
    'package_name @ git+https://gitlab.com/<PRIVATE_ORG>/<PRIVATE_REPO>.git@<COMMIT_ID>'
]

See https://pip.readthedocs.io/en/stable/reference/pip_install/#requirement-specifiers and https://www.python.org/dev/peps/pep-0440/#direct-references

This requires pip install including pip install . and doesn't work with python setup.py install.

In your case:

install_requires=[
    "package @ git+ssh://git@host/repo.git@tag#subdirectory=subdir"
]

setup(install_requires=install_requires)

For example:

install_requires=[
    pycocotools @ git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI
]