307

I'm a bit miffed by the python package installation process. Specifically, what's the difference between packages installed in the dist-packages directory and the site-packages directory?

AjayKumarBasuthkar
  • 2,885
  • 2
  • 23
  • 27
maxm
  • 5,161
  • 7
  • 30
  • 33

3 Answers3

283

dist-packages is a Debian-specific convention that is also present in its derivatives, like Ubuntu. Modules are installed to dist-packages when they come from the Debian package manager into this location:

/usr/lib/python2.7/dist-packages

Since easy_install and pip are installed from the package manager, they also use dist-packages, but they put packages here:

/usr/local/lib/python2.7/dist-packages

From the Debian Python Wiki:

dist-packages instead of site-packages. Third party Python software installed from Debian packages goes into dist-packages, not site-packages. This is to reduce conflict between the system Python, and any from-source Python build you might install manually.

This means that if you manually compile and install Python interpreter from source, it uses the site-packages directory. This allows you to keep the two installations separate, especially since Debian and Ubuntu rely on the system version of Python for many system utilities.

user7610
  • 25,267
  • 15
  • 124
  • 150
jterrace
  • 64,866
  • 22
  • 157
  • 202
  • 30
    Is this accurate?! I install with pip and it goes to dist-packages. (on Deb Squeeze + Py 2.6) – Basel Shishani Apr 04 '12 at 03:24
  • 4
    It's the same for me too. Packages installed with pip or easy_install all goes into dist-packages. – monostop Apr 20 '12 at 15:26
  • 1
    Same here, would I have any problems if I remove all those installations? –  Sep 04 '12 at 21:55
  • Same for me. I am running Ubuntu 12.04 Server, and I install most modules with pip, not with apt-get. They all end up in dist-packages, and site-packages is empty. – Mark E. Haase Sep 14 '12 at 18:10
  • @BaselShishani can you confirm the directories are the same for debian? I only checked Ubuntu – jterrace Sep 18 '12 at 14:30
  • 1
    Same directories for Debian 6.0.10 (still on Python 2.6). – ᴠɪɴᴄᴇɴᴛ Jul 31 '14 at 16:03
  • 2
    If you use virtualenv with pip, pip will install packages in *site-packages* directory. – diabloneo Jun 11 '15 at 09:36
  • On Debian based systems `pip` and `easy_install` will always install into `dist-packages` because they take the location from Python executable which is built-in value (defined on compilation time). If use `virtualenv` with isolated Python executable this location is re-defined there. – Alexey Kamenskiy Aug 05 '16 at 06:51
  • For me: dist-packages (if virtualenv under Centos Linux) and site-packages (virtualenv under mac OS) – jphortense Apr 17 '18 at 12:08
  • @BaselShishani seems accurate, maybe the answer was edited to address your (good) issue. – Timo Sep 15 '21 at 18:53
  • After `apt install python3-pip` I get a `pip` folder in `/usr/lib/python3/dist-packages` – Timo Sep 15 '21 at 19:00
13

Debian (and Ubuntu) has introduced its own convention

# python3 -m site

on Ubuntu Focal gives

sys.path = [
    '/qpid-dispatch',
    '/usr/lib/python38.zip',
    '/usr/lib/python3.8',
    '/usr/lib/python3.8/lib-dynload',
    '/usr/local/lib/python3.8/dist-packages',
    '/usr/lib/python3/dist-packages',
]
USER_BASE: '/root/.local' (doesn't exist)
USER_SITE: '/root/.local/lib/python3.8/site-packages' (doesn't exist)
ENABLE_USER_SITE: True

The convention, as described in the linked mailing list, is that python deb packages installed by the distribution package manager go into /usr/lib/python3/dist-packages and packages installed using sudo pip3 go into /usr/local/lib/python3.8/dist-packages.

If you compile and install your own Python interpreter, it will default to placing itself in /usr/local, with /usr/local/bin/pip3 installs going into /usr/local/lib/pythonX.Y/site-packages.

The point of the Debian convention is to keep the three sets of packages separate:

  1. python packages installed by apt
  2. packages installed by root user with /usr/bin/pip3
  3. packages installed by root user with their own /usr/local/bin/pip3
user7610
  • 25,267
  • 15
  • 124
  • 150
7

dist-packages is the debian-specific directory where apt and friends install their stuff, and site-packages is the standard pip directory.

The problem is -- what happens when different versions of the same package are present in different directories?

My solution to the problem is to make dist-packages a symlink to site-packages:

for d in $(find $WORKON_HOME -type d -name dist-packages); do
  pushd $d
  cd ..
  if test -d dist-packages/__pycache__; then
    mv -v dist-packages/__pycache__/* site-packages/__pycache__/
    rmdir -v dist-packages/__pycache__
  fi
  mv -v dist-packages/* site-packages/
  rmdir -v dist-packages
  ln -sv site-packages dist-packages
  popd
done

(if you are not using gnu tools, remove the -v option).

sds
  • 58,617
  • 29
  • 161
  • 278
  • 4
    I'm a little confused which problem you solve with your solution since the question requiers an explanation – Jürgen K. Apr 20 '19 at 13:57
  • 1
    @JürgenK.: the problem for me was which package is used when there are different versions in different directories. – sds Feb 07 '20 at 14:59
  • Is the solution for python packages in one single place. You run it when you do `apt update` on Debian and new python packages get installed in `dist-packages`? You have a `dist-packages cleaner`, go for a patent.. – Timo May 06 '21 at 06:07
  • one more, I have 2 global `dist-packages`: `/usr/lib/python3/dist-packages` and `/usr/local/lib/python3.8/dist-packages`, should I make one of it? – Timo May 06 '21 at 07:00
  • I got it, that is why you use the loop in case there are more folders. Why do you copy `__pycache__` extra using `if`? I would copy it in one go, no need for `if`. – Timo May 06 '21 at 19:11
  • .. does `rmdir` before `ln` makes sense? If `dist-packages` does not exist, you cannot `ln` or does ln create the linkdir? Wait, according to `man`: `By default, each destination (name of new link) should not already exist` – Timo May 07 '21 at 07:10