12

What is the equivalent of --find-links / -f flag for pip in setup.py.

I know dependency_links exist, but that requires pointing to a specific file, I want something similar to -f that can point to a list of links from which the package can be selected based on version&os.

George
  • 3,521
  • 4
  • 30
  • 75
  • As pointed in my answer `dependency_links` seems to accept "_URLs of web pages that contain direct download links_" according to the _setuptools_ documentation. In order to help you, would you mind updating your question to clarify how it doesn't answer your question? – sinoroc Jan 23 '20 at 11:23

1 Answers1

9

In a setuptools context the dependency_links option should do what you need. According to setuptools documentation, this option accepts:

the URLs of web pages that contain direct download links

for example:

setuptools.setup(
    # ...
    dependency_links=[
        "http://peak.telecommunity.com/snapshots/",
    ],
)

Important note regarding pip:

Since its version 19.0, released on 2019-01-22, pip ignores the setuptools options dependency_links. The solution in a pip context is to use one of the pip install options --index-url, --extra-index-url, or --find-links.

The rationale behind the decision for pip to drop the support of setuptools dependency_links is (in very short): pip should only download from PyPI unless the user themselves explicitly takes the responsibility to allow downloads from alternatives sources by using one of these previously mentioned options. More details can be found for example in this discussion.

sinoroc
  • 18,409
  • 2
  • 39
  • 70
  • Those docs are presumably outdated ? As far as the internet let's me know `dependency_links` is pretty much not working and deprecated since 508 (https://stackoverflow.com/questions/12518499/pip-ignores-dependency-links-in-setup-py) and all examples of it's usage I can find point to a specific repository, not a list. Do you know of any project that actually uses it (successfully) ? – George Aug 30 '19 at 01:58
  • 4
    @George If `pip` is used to install your package then don't set any `dependency_links`. But instead in the documentation of your package, tell the users to use pip with the relevant options `--index-url`, `--extra-index-url`, or `--find-links`.In the end it should be the user's responsibility to choose where to install the dependencies from. The user assumes dependencies come from the official pypi and it is relatively safe. – sinoroc Aug 30 '19 at 09:26
  • 2
    adding additional install instructions isn't an ideal solution in this case, obviously possible, but at that point I can just not use pip or pypi altogether and work around the issue that way, The whole point of using setup.py is to have a standardize install process for use (e.g. what if someone wants to use my package as a dependency) – George Aug 30 '19 at 15:41
  • I looked further into the issue and I don't know of any other way currently than using `pip` with its index options (or skipping pip and installing directy via `setuptools`/`easy_install` and `dependency_links`). Alternatively: look into other package managers such as `conda`. Other idea: in case of an application, maybe consider packaging all the dependencies beforehand, in a `zipapp` for example (see `pex` or `shiv`). – sinoroc Aug 30 '19 at 17:29
  • 2
    @sinroc - I suggest updating this answer to specify that solution only works < pip version 19.0. I know it says it in the link, but it's a bit misleading without the context. I'm also still looking for a supported answer to this question – ZaxR Feb 06 '20 at 19:21
  • @ZaxR The solution in a _pip_ context is to use one of the [`pip install` options `--index-url`, `--extra-index-url`, or `--find-links`](https://pip.pypa.io/en/stable/reference/pip_install/#finding-packages). – sinoroc Feb 06 '20 at 20:58
  • @sinroc - yeah that's how I'm dealing with it at the moment, but that's not an ideal solution for the reason George mentioned. I don't want to force the library users to do any special pip config or add flags. A big benefit of pip is a uniform install experience – ZaxR Feb 06 '20 at 21:03
  • 1
    @ZaxR The rationale behind the decision for _pip_ to drop the support of _setuptools_ `dependency_links` is (in very short): _pip_ only downloads from _PyPI_ and everything else has to be decided explicitly by the user themself. More details can be found for example in [this discussion](https://github.com/pypa/pip/issues/4187#issuecomment-415067034). That decision makes sense to me. – sinoroc Feb 06 '20 at 21:14
  • Thanks - now I can relax knowing the capability was purposefully removed, even if I don't agree with it. If you want to post it, I'll accept your answer https://stackoverflow.com/questions/60099661/setup-py-install-requires-latest-from-private-pypi – ZaxR Feb 06 '20 at 21:16
  • 2
    @ZaxR I still don't understand why it is such a big deal to have to tell the users to use an alternative index. Better this than just forcing the user to download from different locations without letting them know. This is how you build or lose trust. – sinoroc Feb 06 '20 at 21:24
  • I think it's a balance act between ease of use and risk to users. Make it too easy to agree (like we do with user agreements for most things nowadays) and you risk it being a rubber stamp. Make it too hard, and you'll drive people away. I guess my argument is that the balance is a bit too user unfriendly, but ultimately I can see the systemic concerns they're trying to prevent. – ZaxR Feb 06 '20 at 21:50
  • User was already aware of `dependency_links`, I don't see how this answers their question. I think the answer here needs to more clearly communicate that it's the user's responsibility to opt-in with `--index-url`, `--extra-index-url`, or `--find-links`. The package metadata itself is not allowed to make this decision on behalf of the user. The functionality was deprecated/removed for a reason (avoid to accidentally download/install from some random unknown indices) – wim Feb 06 '20 at 22:15
  • @wim It's in the second part of the answer. Is that what you mean? – sinoroc Feb 06 '20 at 22:20
  • Yep, but I think that part should be first and foremost. Any suggestion that pip doesn't support is dead on arrival. The answer also ought to mention [PEP 508 and URL dependencies](https://www.python.org/dev/peps/pep-0508/#examples), which was an attempt to replace a subset of use-cases of those deprecated features. If you don't want to edit in here, I'll add an answer instead .. – wim Feb 06 '20 at 22:34
  • @wim sure, go for it – sinoroc Feb 06 '20 at 22:53
  • @wim Unless I am mistaken PEP508 wouldn't solve much. It would eventually remove the need for `dependency_links` since this info could be added to `install_requires` directly but _pip_/_PyPI_ would still ignore it (for good reasons). @George @ZaxR everyone on the same boat, it really is not that unusual or even remotely problematic to instruct users to point _pip_ at an alternative index. I bet the users are all developers, they can handle it (if it is communicated clearly). Otherwise stick to an old version of _pip_ or even better `easy_install` ;) – sinoroc Feb 06 '20 at 23:23