60

Is there a way to use an extra Python package index (ala pip --extra-index-url pypi.example.org mypackage) with setup.py so that running python setup.py install can find the packages hosted on pypi.example.org?

Guy Avraham
  • 3,482
  • 3
  • 38
  • 50
Jeremy
  • 2,826
  • 7
  • 29
  • 25

8 Answers8

57

If you're the package maintainer, and you want to host one or more dependencies for your package somewhere other than PyPi, you can use the dependency_links option of setuptools in your distribution's setup.py file. This allows you to provide an explicit location where your package can be located.

For example:

from setuptools import setup

setup(
    name='somepackage',
    install_requires=[
        'somedep'
    ],
    dependency_links=[
        'https://pypi.example.org/pypi/somedep/'
    ]
    # ...
)

If you host your own index server, you'll need to provide links to the pages containing the actual download links for each egg, not the page listing all of the packages (e.g. https://pypi.example.org/pypi/somedep/, not https://pypi.example.org/)

bignose
  • 30,281
  • 14
  • 77
  • 110
Heston Liebowitz
  • 1,625
  • 17
  • 12
  • Would be more helpful to include location of this setup.py as well. – Optimus Prime Aug 29 '18 at 07:27
  • How does that handle a `http` index? For instance pip will want `--trusted-host ` for http indexes. – con-f-use Oct 30 '19 at 10:53
  • 10
    as far as I know, dependency links have been deprecated, see e.g.: https://github.com/pypa/setuptools/issues/987 and https://github.com/pypa/pip/issues/4187 – con-f-use Feb 16 '20 at 21:45
  • 8
    The link in the answer says that pip now ignored `dependency_links` but does not say what to use instead. – Troy Daniels Jul 13 '20 at 16:45
  • 5
    Did anybody find a replacement for that? – Matthias Lohr Mar 02 '21 at 11:18
  • This is unfortunately not a robust solution and does not utilize benefits of a dependency manager. The `dependency_links` list needs to list **all** of the dependencies, including transitive ones. Example, you have a dependency on `pandas` and need to use a private mirror of the PyPI. Then you not only need to list `pandas`, but also `numpy`, `python-dateutil`, `pytz` – tony_tiger Jun 11 '21 at 17:05
  • 3
    This does not seem to be working anymore as pypi dropped support fro --process-dependency-links as of v19.0 – rtindru Oct 07 '21 at 22:58
  • See the warning at the top of https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#dependencies-that-aren-t-in-pypi – Lawrence I. Siden Apr 27 '22 at 14:12
17

setuptools uses easy_install under the hood.

It relies on either setup.cfg or ~/.pydistutils.cfg as documented here.

Extra paths to packages can be defined in either of these files with the find_links. You can override the registry url with index_url but cannot supply an extra-index-url. Example below inspired by the docs:

[easy_install]
find_links = http://mypackages.example.com/somedir/
             http://turbogears.org/download/
             http://peak.telecommunity.com/dist/
index-url = https://mypi.example.com
Jean Hominal
  • 16,518
  • 5
  • 56
  • 90
TomDotTom
  • 6,238
  • 3
  • 41
  • 39
  • 2
    thanks for setup.cfg example, worked quite well, unfortunately easy install is deprecated now: https://setuptools.pypa.io/en/latest/deprecated/easy_install.html – Nerxis Oct 27 '21 at 12:51
6

I wanted to post a latest answer to this since both the top answers are obsolete; use of easy_install has been deprecated by setuptools.

https://setuptools.pypa.io/en/latest/deprecated/easy_install.html

Easy Install is deprecated. Do not use it. Instead use pip. If you think you need Easy Install, please reach out to the PyPA team (a ticket to pip or setuptools is fine), describing your use-case.

Please use pip moving forward. You can do one of the following:

  1. provide --index-url flag to pip command
  2. define index-url in pip.conf file
  3. define PIP_INDEX_URL environment variable

https://pip.pypa.io/en/stable/topics/configuration/

rrlamichhane
  • 1,435
  • 2
  • 18
  • 34
  • pip works well during development, but how do I ship a package then? Setuptools bypasses pip, ignores any option like the extra index, and uses the previous easy_install stuff instead. – Eric Buist Apr 26 '23 at 12:28
  • 1
    If we now have to repeatedly rewrite setup.py as setup.cfg then as pyproject.toml, this is really tedious and time consuming and more and more I need tools to translate from one format to the next, because otherwise I cannot work on anything else than trial and error guesswork to figure out what is the current installation format. – Eric Buist Apr 26 '23 at 12:30
4

The following worked for me (develop, not install):

$ python setup.py develop --index-url https://x.com/n/r/pypi-proxy/simple

Where https://x.com/n/r/pypi-proxy/simple is a local PyPI repository.

miku
  • 181,842
  • 47
  • 306
  • 310
  • Not working, and `python setup.py install --help` doesn't have any params relate to `--index-url` – NOZUONOHIGH Aug 05 '19 at 07:03
  • 1
    @NOZUONOHIGH, thanks, I corrected my answer - it was the "develop", not "install", that accepts an index-url flag. – miku Aug 05 '19 at 08:02
  • 1
    If you want to add an extra url like `--extra-index-url` in pip use the `--find-links` option with `setup.py` develop: `python setup.py develop --find-links link` – Mohit Motwani Aug 02 '22 at 11:10
3

Found solution when using Dockerfile:

RUN cd flask-mongoengine-0.9.5 && \
    /bin/echo -e [easy_install]\\nindex-url = https://pypi.tuna.tsinghua.edu.cn/simple >> setup.cfg && \
    python setup.py install

Which /bin/echo -e [easy_install]\\nindex-url = https://pypi.tuna.tsinghua.edu.cn/simple will exists in file setup.cfg:

[easy_install]
index-url = https://pypi.tuna.tsinghua.edu.cn/simple
NOZUONOHIGH
  • 1,892
  • 1
  • 20
  • 20
  • why not include flask-mogoengin-0.9.5/setup.cfg in your source repository? Why create it at docker build time? – Jason Harrison Feb 11 '21 at 00:48
  • 1
    @JasonHarrison It's not create but append. By doing so, we don't need to ADD/COPY a additional modified `setup.cfg` file when build the docker image, one Dockerfile and everything works! – NOZUONOHIGH Feb 18 '21 at 05:50
  • As of Aug 2021, 'index-url' should become 'index_url', at least that's the recommendation I'm getting from setuptools ` UserWarning: Usage of dash-separated 'index-url' will not be supported in future versions. Please use the underscore name 'index_url' instead` – Tad Bumcrot Aug 19 '21 at 18:46
-2

As far as I know, you cant do that. You need to tell pip this, or by passing a parameter like you mentioned, or by setting this on the user environment.

Check my ~/.pip/pip.conf:

[global]
download_cache = ~/.cache/pip
index-url = http://user:pass@localpypiserver.com:80/simple
timeout = 300

In this case, my local pypiserver also proxies all packages from pypi.python.org, so I dont need to add a 2nd entry.

Lovato
  • 2,250
  • 1
  • 16
  • 22
  • this answer appears to be wrong. see the last paragraph in the answer here: http://stackoverflow.com/questions/13353869/use-setuptools-to-install-from-location/13354463?noredirect=1#comment58170495_13354463 – Tommy Feb 05 '16 at 15:40
-2

this worked for me

PIP_INDEX_URL=<MY CUSTOM PIP INDEX URL> pip install -e .

I use setup.py and setup.cfg

ChaitanyaBhatt
  • 1,158
  • 14
  • 11
-8

You can include --extra-index-urls in a requirements.txt file. See: http://pip.readthedocs.org/en/0.8.3/requirement-format.html

Tommy
  • 12,588
  • 14
  • 59
  • 110
  • 1
    I don't think this is correct. The question asks specifically about controlling what `setup.py` does (which we can assume uses `setuptools`) and, IIUC `requirements.txt` is only honoured by `pip` – SpoonMeiser Jul 25 '16 at 10:25
  • 2
    I ended up ditching the setup.py and using this method. – Roman Dec 12 '16 at 12:36
  • 2
    setuptools unable to render `--extra-index-urls` in requirements.txt. The only things it expects is a list of strings with deps version details, etc. `requests>=2.19` – Alex-Bogdanov Jan 15 '19 at 10:41