How to extract dependencies from a PyPi package without downloading it?

Question

I want to get the dependency of a PyPi package remotely without needing to download it completely.

I seem to understand (reading the pip code) that pip when resolving dependencies seems to read the egg once the package has been downloaded.

Is there any other way?

It'll either be in a `requirements.txt` or in `setup.py`, maybe you could download just a single file from the repo, depending on where it's hosted? (i.e. github as opposed to PyPi) [Related Q](https://groups.google.com/forum/?fromgroups=#!topic/comp.lang.python/gclDOiXeN_c) — Alex L, Jan 04 '13 at 09:54
why not, but that needs to be specific to each package, if i don't want to do it by hand for "any" package, it won't be so easy... — Olivier Girardot, Jan 04 '13 at 09:59
You can fetch PyPI package metadata in JSON format at `https://pypi.org/pypi//json`, for example https://pypi.org/pypi/tensorflow/json — Jongbin Park, Jan 02 '20 at 09:21
@JongbinPark Thanks, it it really useful, you can also view a specific version info at `https://pypi.org/pypi///json` additionally, for example https://pypi.org/pypi/pyqt5-tools/5.15.4.3.2/json. — Donghua Liu, Mar 04 '22 at 08:19
@AlexL Sometimes It is not easy or straightforward to find the dependences from `setup.py`, for example https://github.com/altendky/pyqt-tools/blob/main/setup.py#L88. — Donghua Liu, Mar 04 '22 at 08:23

score 14 · Answer 1 · answered Aug 08 '17 at 07:15

Use pipdeptree to view dependencies of installed PyPI packages.

Install:

pip install pipdeptree

Then run:

pipdeptree

You'll see something like that:

Warning!!! Possible conflicting dependencies found:
* Mako==0.9.1 -> MarkupSafe [required: >=0.9.2, installed: 0.18]
  Jinja2==2.7.2 -> MarkupSafe [installed: 0.18]
------------------------------------------------------------------------
Lookupy==0.1
wsgiref==0.1.2
argparse==1.2.1
psycopg2==2.5.2
Flask-Script==0.6.6
  - Flask [installed: 0.10.1]
    - Werkzeug [required: >=0.7, installed: 0.9.4]
    - Jinja2 [required: >=2.4, installed: 2.7.2]
      - MarkupSafe [installed: 0.18]
    - itsdangerous [required: >=0.21, installed: 0.23]
alembic==0.6.2
  - SQLAlchemy [required: >=0.7.3, installed: 0.9.1]
  - Mako [installed: 0.9.1]
    - MarkupSafe [required: >=0.9.2, installed: 0.18]
ipython==2.0.0
slugify==0.0.1
redis==2.9.1

oh come on @Rockallite, the title said "without downloading it".... — user1835157, Aug 14 '23 at 13:02
@user1835157: __technically__ if you already have it installed , you won't need to download it *shrug* — Mr. Developerdude, Aug 30 '23 at 02:03

nealmcb · Answer 2 · 2017-06-16T13:59:10.110

As jinghli notes, there isn't currently a reliable way to get the dependency of an arbitrary PyPi package remotely without needing to download it completely. And in fact the dependencies sometimes depend on your environment, so an approach like Brian's of executing setup.py code is needed in the general case.

The way the Python ecosystem handles dependencies started evolving in the 1990's before the problem was well understood. PEP 508 -- Dependency specification for Python Software Packages sets us on course to improve the situtation, and an "aspirational" draft approach in PEP 426 -- Metadata for Python Software Packages 2.0 may improve it more in the future, in conjunction with the reimplementation of PyPI as Warehouse.

The current situation is described well in the document Python Dependency Resolution.

PyPI does provide a json interface to download metadata for each package. The info.requires_dist object contains a list of names of required packages with optional version restrictions etc. It is often missing, but it is one place to start.

E.g. Django (json) indicates:

{ "info": { ... "requires_dist": [ "bcrypt; extra == 'bcrypt'", "argon2-cffi (>=16.1.0); extra == 'argon2'", "pytz" ], ... }

The `requires_dist` block doesn't show up for the linked JSON from pypi anymore :( — Ash Berlin-Taylor, Apr 27 '17 at 12:48
Thanks @AshBerlin. I changed the example, since jupyter no longer has a `requires_dist` — nealmcb, Jun 16 '17 at 14:00
I think it does (again?)? A sample [here](https://pypi.org/pypi/pymc/json). You can just search for `requires_dist`. This answer should be the accepted one, nice explanation @nealmcb! — Jorrick Sleijster, Jul 11 '22 at 18:19

score 8 · Accepted Answer · edited May 23 '17 at 12:25

8

I've just needed to find a way to do this and this is what I came up with (stolen from pip).

def dist_metadata(setup_py):
    '''Get the dist object for setup.py file'''

    with open(setup_py) as f:
        d = f.read()

    try:
        # we have to do this with current globals else
        # imports will fail. secure? not really. A
        # problem? not really if your setup.py sources are 
        # trusted
        exec d in globals(), globals()
    except SystemExit:
        pass

    return distutils.core._setup_distribution

https://stackoverflow.com/a/12505166/3332282 answers why the exec incantation is subtle and hard to get right.

edited May 23 '17 at 12:25

Community

1
1

answered Feb 20 '14 at 10:43

Brian Stengaard - Podio

876
5
8

Thanks - tricky! But note that you need `import distutils` at the top, and the caller will want to look at `dist_metadata(setup_py).install_requires`, which returns a list of package names. That leaves the question of how to get the `setup.py` without downloading the whole package, as OP requested. – nealmcb Apr 11 '17 at 14:40
This does not work as-is with Python3. You need to change `exec d in globals(), globals()` to `exec(d, globals())` and to get requirements looks at the `dist_metadata(setup_py).get_requires()` method. – mickours Apr 17 '19 at 07:48

jinghli · Answer 4 · 2013-01-04T15:24:00.583

3

Sadly, pip doesn't have this function. The metadata available for packages on PyPI does not include information about the dependencies.

Normally, you can find the detailed dependency from the README file from project website.

pip search can give some information about the package. It can tell you what is it based on.

$ pip search flask
Flask     - A microframework based on Werkzeug, Jinja2 and good intentions

edited Jan 04 '13 at 15:24

answered Jan 04 '13 at 11:22

jinghli

617
4
11

4

Thanks for the info. I'm really surprised! I assumed, based on how Debian/Ubuntu package management works, that this would be a basic piece of data to require from packages in a standard way, so that tools like [pypi-data](https://github.com/nathforge/pypi-data) could show the full dependency tree. Though I dare say it's complicated.... – nealmcb Apr 11 '17 at 14:27

score 1 · Answer 5 · answered Mar 14 '23 at 11:57

1

You can fetch PyPI package metadata in JSON format using a URL like: https://pypi.org/pypi/<package name>/json.

Example: https://pypi.org/pypi/tensorflow/json

From a comment on the OP that was the right answer for me.

answered Mar 14 '23 at 11:57

alextes

1,817
2
15
22

How to extract dependencies from a PyPi package without downloading it?

5 Answers5

Linked