8

I'm writting a python app that depends on another one that is hosted on a github repository (never in pypi) for development reasons.

Lets call them:

  • App being written: AppA
  • App in github: AppB

In App A, the setup.py is like:

# coding=utf-8
import sys
try:
    from setuptools import setup, find_packages
except ImportError:
    import distribute_setup
    distribute_setup.use_setuptools()
    from setuptools import setup, find_packages

setup(
    ...
    install_requires=[
        # other requirements that install correctly
        'app_b==0.1.1'
    ],
    dependency_links=[
        'git+https://github.com/user/app_b.git@0.1.1#egg=app_b-0.1.1'
    ]
)

Now AppA is being built by Jenkins CI with every push and I get a failure because of the next error is thrown:

error: Download error for git+https://github.com/user/app_b.git@0.1.1: unknown url type: git+https

Funny thing is that this only happens in Jenkins, it works perfectly on my computer. I tried both of the other SSH urls that github gives and those are not even considered for download.

Now, AppA is included in the requirements file of a project also being built by Jenkins, so installing the dependencies manually via pip install AppA pip install AppB is not an option, the dependencies are automatically installed by being included in the requirements.txt.

Is there any way to make pip and git with github urls work together?

Any help will be very appreciated :)

Thanks in advance!

Bibhas Debnath
  • 14,559
  • 17
  • 68
  • 96
Gerard
  • 9,088
  • 8
  • 37
  • 52
  • Are you sure Jenkins uses pip? If yes, is it a version that supports this feature? – wRAR Feb 06 '13 at 22:13
  • @Gerard what version of pip does your Jenkins box use? `pip --version` should give you that information. Off the cuff, it looks like pre-0.8.2, which did not support the https scheme. – John Szakmeister Feb 07 '13 at 12:01

3 Answers3

12

The problem is not with pip, is with setuptools. The responsible for the setup() call is setuptools package (setuptools or distribute project).

Neither setuptools or distribute understand that kind of url, they understand tarballs/zip files.

Try pointing to Github's download url - usually a zip file.

Your dependency_links entry is probably going to look like:

dependency_links=[
    'https://github.com/user/app_b/archive/0.1.1.zip#egg=app_b-0.1.1'
]

For more information take a look at http://peak.telecommunity.com/DevCenter/setuptools#dependencies-that-aren-t-in-pypi

Hugo Lopes Tavares
  • 28,528
  • 5
  • 47
  • 45
  • In addition to this answer, in my case the package could never be installed if such package within `requires` was in the form `package==version`, but it worked using `package`; that is, only the package name without version – Gerard Apr 04 '13 at 09:33
  • how could you add a private repository's branch as a dependency link? This is after all one of the main reasons to use git+[https|ssh] style requirements – patzm Sep 07 '18 at 15:06
2

From pip documentation - 

pip currently supports cloning over git, git+http and git+ssh:

git+git://git.myproject.org/MyProject#egg=MyProject
git+http://git.myproject.org/MyProject#egg=MyProject
git+ssh://git.myproject.org/MyProject#egg=MyProject

Try replacing git+https with git+git.

Bibhas Debnath
  • 14,559
  • 17
  • 68
  • 96
  • Sorry this didn't work. It works locally but not within Jenkins :( I'm trying to figure something out. thanks anyway – Gerard Feb 06 '13 at 21:45
  • `git+https` should be supported, despite the documentation. At least according to the [source](https://github.com/pypa/pip/blob/develop/pip/vcs/git.py#L17). That appeared all the way back in pip 0.8.2. – John Szakmeister Feb 07 '13 at 11:59
  • I'm just saying what the error is showing - `unknown url type: git+https`. Must be some old version of pip. – Bibhas Debnath Feb 07 '13 at 16:02
0

I had the same issue in 2019, but due to different reasons. dependency_links are not supported in pip anymore (tested with pip>=20.0.0). For my case I solved this issue using install_requirements and defining a direct reference (see pip manual direct reference).

...
install_requirements = [
    <dependencyname> @ git+<url of dependency repository>@<branchname or tag>
]

I made an puplic example repository named thepackage at https://gitlab.rhrk.uni-kl.de/scheliga/thepackage for more details.

D. Scheliga
  • 1