pip install dependency links

Question

I am using python version 2.7 and pip version is 1.5.6.

I want to install extra libraries from url like a git repo on setup.py is being installed.

I was putting extras in install_requires parameter in setup.py. This means, my library requires extra libraries and they must also be installed.

...
install_requires=[
    "Django",
    ....
],
...

But urls like git repos are not valid string in install_requires in setup.py. Assume that, I want to install a library from github. I have searched about that issue and I found something which I can put libraries such that in dependency_links in setup.py. But that still doesn't work. Here is my dependency links definition;

dependency_links=[
    "https://github.com/.../tarball/master/#egg=1.0.0",
    "https://github.com/.../tarball/master#egg=0.9.3",
],

The links are valid. I can download them from a internet browser with these urls. These extra libraries are still not installed with my setting up. I also tried --process-dependency-links parameter to force pip. But result is same. I take no error when pipping.

After installation, I see no library in pip freeze result in dependency_links.

How can I make them to be downloaded with my setup.py installation?

Edited:

Here is my complete setup.py

from setuptools import setup

try:
    long_description = open('README.md').read()
except IOError:
    long_description = ''

setup(
    name='esef-sso',
    version='1.0.0.0',
    description='',
    url='https://github.com/egemsoft/esef-sso.git',
    keywords=["django", "egemsoft", "sso", "esefsso"],
    install_requires=[
        "Django",
        "webservices",
        "requests",
        "esef-auth==1.0.0.0",
        "django-simple-sso==0.9.3"
    ],
    dependency_links=[
        "https://github.com/egemsoft/esef-auth/tarball/master/#egg=1.0.0.0",
        "https://github.com/egemsoft/django-simple-sso/tarball/master#egg=0.9.3",
    ],

    packages=[
        'esef_sso_client',
        'esef_sso_client.models',
        'esef_sso_server',
        'esef_sso_server.models',
    ],
    include_package_data=True,
    zip_safe=False,
    platforms=['any'],
)

Edited 2:

Here is pip log;

Downloading/unpacking esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
  Getting page https://pypi.python.org/simple/esef-auth/
  Could not fetch URL https://pypi.python.org/simple/esef-auth/: 404 Client Error: Not Found
  Will skip URL https://pypi.python.org/simple/esef-auth/ when looking for download links for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
  Getting page https://pypi.python.org/simple/
  URLs to search for versions for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0):
  * https://pypi.python.org/simple/esef-auth/1.0.0.0
  * https://pypi.python.org/simple/esef-auth/
  Getting page https://pypi.python.org/simple/esef-auth/1.0.0.0
  Could not fetch URL https://pypi.python.org/simple/esef-auth/1.0.0.0: 404 Client Error: Not Found
  Will skip URL https://pypi.python.org/simple/esef-auth/1.0.0.0 when looking for download links for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
  Getting page https://pypi.python.org/simple/esef-auth/
  Could not fetch URL https://pypi.python.org/simple/esef-auth/: 404 Client Error: Not Found
  Will skip URL https://pypi.python.org/simple/esef-auth/ when looking for download links for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
  Could not find any downloads that satisfy the requirement esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
Cleaning up...
  Removing temporary dir /Users/ahmetdal/.virtualenvs/esef-sso-example/build...
No distributions at all found for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
Exception information:
Traceback (most recent call last):
  File "/Users/ahmetdal/.virtualenvs/esef-sso-example/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Users/ahmetdal/.virtualenvs/esef-sso-example/lib/python2.7/site-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Users/ahmetdal/.virtualenvs/esef-sso-example/lib/python2.7/site-packages/pip/req.py", line 1177, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/Users/ahmetdal/.virtualenvs/esef-sso-example/lib/python2.7/site-packages/pip/index.py", line 277, in find_requirement
    raise DistributionNotFound('No distributions at all found for %s' % req)
DistributionNotFound: No distributions at all found for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)

It seems, it does not use the sources in dependency_links.

score 34 · Answer 1 · answered Feb 20 '19 at 19:01

34

The --process-dependency-links option to enable dependency_links was removed in Pip 19.0.

Instead, you can use a PEP 508 URL to specify your dependency, which is supported since Pip 18.1. Here's an example excerpt from setup.py:

install_requires=[
    "numpy",
    "package1 @ git+https://github.com/user1/package1",
    "package2 @ git+https://github.com/user2/package2@branch1",
],

Note that Pip does not support installing packages with such dependencies from PyPI and in the future you will not be able to upload them to PyPI (see news entry for Pip 18.1).

answered Feb 20 '19 at 19:01

Mihai Capotă

2,271
2
32
24

Unlike dependency_links, this solution worked for me. – user4052054 Jul 22 '19 at 20:50
1

You still need (at least in my case) to keep dependency_links to install with `python setup.py install` – educolo Oct 06 '19 at 22:39
@educolo 's suggestion to combine the PEP508 syntax with `dependency_links` is currently the only way to get private repo packages to work in `pip install` as well as `setuptools` (which is used for nested dependencies in `pip`): `pip` 19.0 and later no longer support `dependency_links`, while `setuptools` [still prefers PyPI over PEP 508 specified URLs in `install_requires`](https://github.com/pypa/setuptools/issues/787). – Tomas Dec 31 '19 at 10:54
It is not true that dependency_links was removed, follow the link you posted and reread the notes. – Pykler Jan 02 '20 at 06:07

Pykler · Answer 2 · 2017-01-27T19:56:01.060

19

Pip removed support for dependency_links a while back. The latest version of pip that supports dependency_links is 1.3.1, to install it

pip install pip==1.3.1

your dependency links should work at that point. Please note, that dependency_links were always the last resort for pip, ie. if a package with the same name exists on pypi it will be chosen over yours.

Note, https://github.com/pypa/pip/pull/1955 seems to start allowing dependency_links, pip kept it, but you might need to use some command line switches to use a newer version of pip.

EDIT: As of pip 7 ... they rethought dep links and have enabled them, even though they haven't removed the deprecation notice, from the discussions they seem to be here to stay. With pip>=7 here is how you can install things

pip install -e . --process-dependency-links --allow-all-external

Or add the following to a pip.conf, e.g. /etc/pip.conf

[install]
process-dependency-links = yes
allow-all-external = yes
trusted-host =
    bitbucket.org
    github.com

EDIT

A trick I have learnt is to bump up the version number to something really high to make sure that pip doesn't prefer the non dependency link version (if that is something you want). From the example above, make the dependency link look like:

"https://github.com/egemsoft/django-simple-sso/tarball/master#egg=999.0.0",

Also make sure the version either looks like the example or is the date version, any other versioning will make pip think its a dev version and wont install it.

edited Jan 27 '17 at 19:56

answered Apr 08 '15 at 19:54

Pykler

14,565
9
41
50

Looks like this is no more the case? dependency_links is supported in latest pip as far as I can see. – yelsayed Jan 11 '17 at 23:01
You still see the warning, but dependency_links are going to stay for a while. You still need the config as far as I know, see the note in the edit section of the answer. – Pykler Jan 16 '17 at 15:18
4

maybe you should remove informations about pip version 1.3.1 it's very old – thibault ketterer Aug 23 '17 at 15:23
@thibaultketterer the answer has edits, it is still correct as is no need to edit it – Pykler Aug 25 '17 at 14:09
1

Thank you very much for the egg= version trick, that was the missing piece in the puzzle! – TheDiveO Aug 30 '17 at 07:03
An update on the verge of 2020: pip 1.3.1 is terribly outdated (>5 years) and should not be used, pip 19 [no longer supports --process-dependency-links](https://pip.pypa.io/en/stable/news/#id146) and there are better alternatives to the version number hack in other answers. @mihai provides a more future-proof answer [here](https://stackoverflow.com/a/54793503/1580660). – Tomas Dec 31 '19 at 10:29
@Tomas there is an update to mention newer versions of pip, and when they decided to undo the rash decision they made (that is seen with pip >1.3.1 < 7) – Pykler Jan 02 '20 at 06:05
@Pykler I've read an [issue report](https://github.com/pypa/pip/issues/5898) on GitHub stating why `dependency_links` should be reinstated, but no concise updates/decisions yet. Could you point me to that statement? – Tomas Jan 03 '20 at 08:34

mjallday · Accepted Answer · 2014-10-01T15:33:58.770

13

You need to make sure you include the dependency in your install_requires too.

Here's an example setup.py

#!/usr/bin/env python
from setuptools import setup

setup(
    name='foo',
    version='0.0.1',
    install_requires=[
        'balog==0.0.7'
    ],
    dependency_links=[
        'https://github.com/balanced/balog/tarball/master#egg=balog-0.0.7'
    ]
)

Here's the issue with your example setup.py:

You're missing the egg name in the dependency links you setup.

You have

https://github.com/egemsoft/esef-auth/tarball/master/#egg=1.0.0.0

You need

https://github.com/egemsoft/esef-auth/tarball/master/#egg=esef-auth-1.0.0.0

edited Oct 01 '14 at 15:33

answered Sep 26 '14 at 15:16

mjallday

9,796
9
51
71

1

It says `Could not find any downloads that satisfy the requirement my-extra-libraray==1.0.0.0 (from my-installed-library-currently==1.0.0.0)` – Ahmet DAL Sep 28 '14 at 17:19
@AhmetDAL you have both the dependency links and install requires specified? can you post the complete `setup.py` that you are using? – mjallday Sep 28 '14 at 18:33
1

result is still same. Here is my [setup.py](https://github.com/egemsoft/esef-sso/blob/master/setup.py). Can you try with `pip install git+https://github.com/egemsoft/esef-sso.git`. You are gonna probably see the result. – Ahmet DAL Oct 02 '14 at 07:02
1

`django-simple-sso` should have also been from my forked source. It is coming from original source instead. I think, `install_requires` does not use `dependency_links` as source. – Ahmet DAL Oct 02 '14 at 07:09

score 3 · Answer 4 · edited Jul 22 '19 at 03:21

I faced a similar situation where I want to use shapely as one of my package dependency. Shapely, however, has a caveat that if you are using windows, you have to use the .whl file from http://www.lfd.uci.edu/~gohlke/pythonlibs/. Otherwise, you have to install a C compiler, which is something I don't want. I want the user to simply use pip install mypackage instead of installing a bunch of other stuffs.

And if you have the typical setup with dependency_links

setup(
  name = 'streettraffic',
  packages = find_packages(), # this must be the same as the name above
  version = '0.1',
  description = 'A random test lib',
  author = 'Costa Huang',
  author_email = 'Costa.Huang@outlook.com',
  install_requires=['Shapely==1.5.17'],
  dependency_links = ['http://www.lfd.uci.edu/~gohlke/pythonlibs/ru4fxw3r/Shapely-1.5.17-cp36-cp36m-win_amd64.whl']
)

and run pip install ., it is simply going to pick the shapely on Pypi and cause trouble on Windows installation. After hours of researching, I found this link Force setuptools to use dependency_links to install mysqlclient and basically use from setuptools.command.install import install as _install to manually install shapely.

from setuptools.command.install import install as _install
from setuptools import setup, find_packages
import pip

class install(_install):
  def run(self):
    _install.do_egg_install(self)

    # just go ahead and do it
    pip.main(['install', 'http://localhost:81/Shapely-1.5.17-cp36-cp36m-win_amd64.whl'])

setup(
  name = 'mypackage',
  packages = find_packages(), # this must be the same as the name above
  version = '0.1',
  description = 'A random test lib',
  author = 'Costa Huang',
  author_email = 'test@outlook.com',
  cmdclass={'install': install}
)

And the script works out nicely. Hope it helps.

pip install dependency links

Edited:

Edited 2:

4 Answers4

Linked