Python: programmatically running "pip list"

Question

I'm writing a bit of code that will report and reconcile differences between two pip-managed python installations.

How can I programmatically get the information provided by pip list without making a subprogram invocation of pip?

Answer 1

The top answers as of 2/1/2019 are outdated and no longer work with newer versions of pip.

But no worries - it's still possible to get a list of packages programmatically:

Options:

A. _internal.main

from pip import _internal
_internal.main(['list'])

This will print out three columns with Package. Version, and Location

Note that usage of pip's internal api is not recommended.

B. pkg_resources

import pkg_resources
print([p.project_name for p in pkg_resources.working_set])
# note that this is same as calling pip._vendor.pkg_resources.working_set

C. iter_modules

Takes a long time to execute (~300ms on computer w/ I5 CPU, SSD, & 8 gigs ram). The benefit is that it will have a far more extensive list of modules and it will output importable names.

Ex: python-dateutil is imported as dateutil, but iter_modules will give you the importable name: dateutil

from pkgutil import iter_modules
print([p.name for p in iter_modules()])

D. Call pip in command line via subprocess

The solution to this is trivial and I'll leave this as an exercise to the reader

aka I'm too lazy to do this, good luck! :D

Answer 2

Update for Python 3.6 and Pip 19.0.1

> from pip._internal.utils.misc import get_installed_distributions
> p = get_installed_distributions()
> pprint.pprint(p)

[wheel 0.32.3 (/usr/local/lib/python3.7/site-packages),
 wcwidth 0.1.7 (/usr/local/lib/python3.7/site-packages),
 virtualenv 16.0.0 (/usr/local/lib/python3.7/site-packages),
 virtualenv-clone 0.3.0 (/usr/local/lib/python3.7/site-packages),
 urllib3 1.24.1 (/usr/local/lib/python3.7/site-packages),
 typing 3.6.6 (/usr/local/lib/python3.7/site-packages),
 terminaltables 3.1.0 (/usr/local/lib/python3.7/site-packages),
 ...

Original Answer

Pip is just python module, so just import it and call list:

import pip

pip.main(['list'])

# you can get details on package using show:

pip.main(['show', 'wheel'])

Ok so there is better way:

pip.utils.get_installed_distributions()

returns you list of packages installed.

packages = pip.utils.get_installed_distributions()

p = packages[0]

p.project_name 
p.version
p.egg_name
p.location

You can see what pip list is doing from the source code here

Also get_installed_distributions accept whole bunch of parameters to return only local packages (from current virtualenv) etc. Please see help here.

There is also underlying low level command from _vendor module:

list(pip._vendor.pkg_resources.working_set)

However get_installed_distributions provide simplier api.

Answer 3

Use os module or system module

import os 
import subprocess as su
os.system("pip list")
su.call(["pip","list"])

Answer 4

python -m pip list (robust method)

import subprocess
import sys

def pip_list():
    args = [sys.executable, "-m", "pip", "list"]
    p = subprocess.run(args, check=True, capture_output=True)
    return p.stdout.decode()

print(pip_list())

As @Aaron mentions:

The officially recommended way to install packages from a script is by calling pip's command-line interface via a subprocess. Most other answers presented here are not supported by pip. Furthermore since pip v10, all code has been moved to pip._internal precisely in order to make it clear to users that programmatic use of pip is not allowed.

Use sys.executable to ensure that you will call the same pip associated with the current runtime.

Answer 5

After testing some solution here that is either very slow, deprecated, or return error in my Python 3.10, I'm using this solution:

Note

This function can retrieve all installed package with version installed.

import pkg_resources

def get_installed_packages():
    installed_packages = []
    for package in pkg_resources.working_set:
        installed_packages.append(package.key)
    return installed_packages

def get_package_version(package_name):
    try:
        return pkg_resources.get_distribution(package_name).version
    except pkg_resources.DistributionNotFound:
        return None

# Get a list of all installed packages
installed_packages = get_installed_packages()

# Iterate over the installed packages and get their versions
package_versions = {}
for package_name in installed_packages:
    version = get_package_version(package_name)
    package_versions[package_name] = version

# Print the package versions
for package_name, version in package_versions.items():
    print(f"{package_name} - {version}")

Example output:

xlsxwriter - 3.0.9
argcomplete - 2.0.0
comm - 0.1.2
debugpy - 1.6.6

Combined with regex to list package name, we can check all installed version from requirements that is actually installed in the machine.

import re

def extract_package_names(file_path):
    with open(file_path, 'r') as file:
        requirements = file.readlines()

    package_names = []
    for requirement in requirements:
        match = re.search(r'^([\w.-]+)', requirement)
        if match:
            package_names.append(match.group(1))

    return package_names

# Example usage
file_path = 'requirements.txt'
package_names = extract_package_names(file_path)
print(package_names)

Example output:

['numpy', 'pandas', 'xlsxwriter']

The two combined:

file_path = 'requirements.txt'
package_names = extract_package_names(file_path)

packages = {}
for package_name in package_names:
    packages[package_name] = get_package_version(package_name)
packages

Note

DISCLAIMER: This code is written with the help of ChatGPT 3.5

Answer 6

For completeness, here's vittore's pip.main() idea fleshed out with the capture of stdout. Of course using get_installed_distributions() is the preferred solution.

import contextlib
@contextlib.contextmanager
def capture():
    import sys
    from cStringIO import StringIO
    oldout,olderr = sys.stdout, sys.stderr
    try:
        out=[StringIO(), StringIO()]
        sys.stdout,sys.stderr = out
        yield out
    finally:
        sys.stdout,sys.stderr = oldout, olderr
        out[0] = out[0].getvalue()
        out[1] = out[1].getvalue()

with capture() as out:
    import pip
    pip.main(['list'])

print out
    ['awscli (1.7.45)\nboto (2.38.0) ...