pip freeze: show only packages installed via pip

Question

I want to know which python packages are installed via pip and which are installed via rpm.

I run outside any virtualenv, and want to know if there are some packages installed via pip.

Background: Our policy is to use RPM at "root level". I want to find places where the policy was broken.

Your situation is, you have multiple virtualenvs, and you want to check packages installed via pip from the root level? — Laszlowaty, May 26 '16 at 13:21
If `pip list` returns a list of all installed packages, you can use `rpm -qa` to find the packages installed via `rpm` and the difference of the two will be the packages not conforming to your policy. Would the above work for you ? — Jacques Gaudin, May 26 '16 at 13:37
I guess both names are not always identical (byte per byte), but only similar (equal looking for the human brain). One example: pip freeze tells it "python-dateutil", RPM name is the same. pip freeze tells me "Pillow", as RPM it is called python-Pillow... — guettli, May 30 '16 at 09:27
In general, no I don't think it's possible. You can check `$HOME/.cache/pip` or similar for root, but that cannot protect you from sequence `pip install x-1.2.3; apt-get install python-x-1.2.3` — Dima Tisnek, May 30 '16 at 10:24

score 2 · Answer 1 · edited May 23 '17 at 11:52

Assumptions:

I'm not sure about red-hat, but for debian/ubuntu.
I'm assuming you are using system python.
I don't think it matters, but you might have to check pip install --user <package_name> for local user package installs.

By default on debian system installed packages are installed at:

/usr/lib/python2.7/dist-packages/

And pip installed packages are installed at:

/usr/bin/local/python2.7/dist-packages

To see all the installation paths you can run inside your python shell:

import site; site.getsitepackages()
['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']

As per the pip freeze docs-l will show you any local installs of packages (i.e. not global packages) However, you need to be in the correct environment.

pip freeze -l

If Virtualenvs come into play: They will use site-packages directories.

locate -r '/site-packages$'

Also note any packages installed into a different directory will not be located at all via this method: Install a Python package into a different directory using pip?

Final trick, Check the exact install path in pip using pip show. Effectively, get just the names from pip, pipe that into a pip show and filter the output for the Name -> Location map.

pip freeze | awk '{split($0,a,"="); print a[1]}' | xargs -P 5 -I {} pip show {} | grep 'Name\|Location'

`pip freeze -l` is new to me. I looked at the diff between freeze with and without `-l` (inside a virtualenv). Works fine to detect "inside or outside" virtualenv. But it does not help to decide"from-rpm or from pip". — guettli, Jun 01 '16 at 09:58
Does rpm install into a different location compared to pip on your setup? for instance `/usr/lib/` vs `/usr/bin/local/` or is it the same? — Luke Exton, Jun 01 '16 at 10:01
no unfortunately not. It installs into the same location: `/usr/lib/python2.7/site-packages/` — guettli, Jun 01 '16 at 10:06
I have added a one liner that will hopefully help identify the exact install paths. It should also be noted that a lot of these packages installed will come by default on the system, not even installed via an rpm. — Luke Exton, Jun 01 '16 at 17:04

score 2 · Accepted Answer · edited Jun 05 '16 at 12:22

How about turn the question around slightly, and just check what belongs to rpms and what doesn't. Try:

import os, sys, subprocess, glob

def type_printed(pth, rpm_dirs=False):

    if not os.path.exists(pth):
        print(pth + ' -- does not exist')
        return True        
    FNULL = open(os.devnull, 'w')
    if rpm_dirs or not os.path.isdir(pth):
        rc = subprocess.call(['rpm', '-qf', pth], stdout=FNULL, stderr=subprocess.STDOUT) 
        if rc == 0:
            print(pth + ' -- IS RPM')
            return True 
        print(pth + ' -- NOT an RPM')
        return True 
    return False 


for pth in sys.path:
    if type_printed(pth):
        continue 
    contents = glob.glob(pth + '/*') 
    for subpth in contents:
        if type_printed(subpth, rpm_dirs=True):
            continue
        print(subpth + ' -- nothing could be determined for sure')

And pipe the output through something like

grep -e '-- NOT' -e '-- nothing could be determined'

Yes, great. This way I could find the stuff that was installed via pip as root. I like pip, but only in virtualenv :-) — guettli, Jun 05 '16 at 12:23

pip freeze: show only packages installed via pip

2 Answers2