9

There are quite a few people wondering for an alternative to dependency links in the setup.py (activated with the pip flag --process-dependency-links): What is the alternative to using --process-dependency-links with pip, Depend on git repository in setup.py. Basically, I got bitten by the deprecation warning:

"DEPRECATION: Dependency Links processing has been deprecated and will be removed in a future release."

Some people have suggested using requirements.txt, however that is not an alternative as it is meant to provide an entire environment, usually more associated with development. The install_requires is supposed to provide a sort of minimum set of libraries that are necessary to work with the standard functionality, so that when you do something like pip install [LIBRARY], everything needed is installed, without any further pip install -r requirements.txt (I am referring to cases in which the LIBRARY paramater of pip install [LIBRARY] would come in the form of a URL like git+http:\\github.com\username\repo.git).

My issue with the deprecation is that I cannot reference internal/private packages, but I can also see how this could be a problem if it is required to reference a particular commit or branch in git (at least I know I had done this in the past).

All that said, the use of dependency_links is complicated, e.g. the syntaxis is not always clear, there exist several ways of specifying urls, and people tend to forget that they have to put the name-version of the library in both the dependency_links and the install_requires lists. I would like to hear that this deprecation is in favor of an improvement, but doesn't seem to be the case

So, to summarize, what is the reason for deprecating dependency links? Is the deprecation of dependency links in favour of a better alternative? It doesn't seem that there is an alternative

toto_tico
  • 17,977
  • 9
  • 97
  • 116
  • "`requirements.txt` ... is meant to provide an entire environment." Are you saying that `requirements.txt` usually includes dependencies that aren't strictly necessary? Can't you just edit those out? You could always have two files if you still want one that contains optional extra modules for development. I'm just confused why you don't consider it an alternative (but I don't fully understand dependency links). – Arthur Tacca Sep 20 '17 at 11:16
  • @ArthurTacca, as you said, it is common to have a `requirements.txt` and `requirements_dev.txt` (or similar). One is for developers, and the other lists what is necessary to run all the functionality of the library. In any case, you need to use `pip install -r requirements.txt`. The idea with the `install_requires` is that when you, for example, use `pip install pandas`, then all the dependencies are installed with it. In other words, after or before installing `pandas` you don't have to do `pip install -r requirements.txt`. – toto_tico Sep 20 '17 at 12:12
  • @ArthurTacca dependency links just extend the scope of libraries beyond libraries available in `pip install ...` – toto_tico Sep 20 '17 at 12:13
  • Just fighting myself with the same problem. I think I will just run my own pypi server as mentioned [here](https://github.com/pypa/pip/issues/3610#issuecomment-341706690). – Niko Föhr Jan 08 '18 at 12:44

1 Answers1

4

PEP 508 URL dependencies are the alternative for dependency-links. You can find more details about that in my related answer.

What is the reason for deprecating dependency links?

Security. When dependency links are enabled, pip can be made to fetch arbitrary URLs from the internet and run code from them -- something that is obviously not a good idea.

You can read more about it in the original thread proposing this: https://mail.python.org/pipermail/distutils-sig/2013-October/022937.html

pradyunsg
  • 18,287
  • 11
  • 43
  • 96
  • Well, the problem is that deprecating something doesn't make it secure, does it?. I mean, it is still there and it will be secure until it is removed (for now, it is just confusing people). The underlying reason seems to be that there is a sort of issue between Pypi and pip. In principle, there are many ways of solving this (e.g. not accepting the parameter in Pypi, whitelists, accepting local directories, etc.), and yet this has been there since 2013, IMHO people deserve, for the time being, an explanation. – toto_tico May 29 '18 at 20:20
  • It doesn't but it discourages people from using it. I understand that it's providing functionality that does not have an alternative (yet!). Moving people to use PEP 508 URL requirements is a good idea and with some period to allow people to transition, should work out well. – pradyunsg May 30 '18 at 06:08
  • Aside: Not accepting the parameter on PyPI is not a feasible task since that involves running arbitrary code -- something that one shouldn't expect any service to just do without a _lot_ of investment in ensuring it stays secure -- that sort of investment, PyPI just doesn't have. – pradyunsg May 30 '18 at 06:10
  • @pradyunsg, where has the PEP 508 URL dependencies functionality been implemented? How does one use it? – Hooloovoo13 Jul 22 '18 at 12:32
  • As it stands, currently it's only usable as directly specified dependencies. Basically, directly via the command line or in a requirements file. The next non-bugfix release of pip will include the more complete support I mentioned above -- using it to specify dependencies. (PR for that: https://github.com/pypa/pip/pull/5571) – pradyunsg Jul 29 '18 at 11:05
  • 1
    @pradyunsg, it seems your PR provides something that people have been waiting for a long time in PIP. Well done! – Hooloovoo13 Jul 31 '18 at 10:50