64

How do you prevent PIP from re-downloading previously downloaded packages? I'm testing the install of matplotlib, an 11MB package that depends on several distro-specific packages. Everytime I run pip install matplotlib, it re-downloads matplotlib. How do I stop this?

Cerin
  • 60,957
  • 96
  • 316
  • 522

3 Answers3

114

NOTE: Only wheels downloaded over HTTPS are cached. If you are using a custom repo over plain old HTTP, the cache is disabled.

For new Pip versions:

Newer Pip versions by default now cache downloads. See this documentation:

https://pip.pypa.io/en/stable/topics/caching/

For old Pip versions:

Create a configuration file named ~/.pip/pip.conf, and add the following contents:

[global]
download_cache = ~/.cache/pip

In one command:

printf '[global]\ndownload_cache = ~/.cache/pip\n' >> ~/.pip/pip.conf
Flimm
  • 136,138
  • 45
  • 251
  • 267
  • 1
    Cool, easy and do not need to remember adding the parameter each time. I wasted a LOT of Gb's re-downloading shit. Thank you. – m3nda Jan 04 '16 at 08:58
  • Thank you, if you use pip3 to download requirements and get yellow text warning, you can mkdir ~/.cache/pip then the warning will go away. – 水清木华 Sep 25 '17 at 09:58
  • I added a note to use HTTPS. There's a setting `--trusted-host option` that can enable caching over HTTP for a specific domain in pip 21.3 (2021-10-11). – user5994461 Nov 11 '21 at 10:02
  • By the way, what is the life time (TTL) of a package in pip cache ? – Muhammad Faizan-Ul-Haq Sep 21 '22 at 06:55
63

You can use a specific environment variable PIP_DOWNLOAD_CACHE and make it point to a directory where your packages will be stored. If they are to be installed again, they will be taken from this directory.

There seems to be also an additional option for PIP pip --download-cache which ought to do something similar, but I have never tried it myself. For your example, to avoid re-downloading matplotlib every time, you would do the following:

pip install --download-cache /path/to/pip/cache matplotlib

Does that answer your question?

Charles Menguy
  • 40,830
  • 17
  • 95
  • 117
  • I'm not sure if PIP does it too, but with `easy_install` if you have the package as a `.tar` (or presumably zip?) file in the local directory it will try to use that one first. – Wayne Werner Apr 26 '12 at 16:21
  • 2
    Do look at [pip-accel](https://pypi.python.org/pypi/pip-accel) as well. It's a new and better solution to this problem. – qris Oct 02 '14 at 13:47
  • 11
    as of pip 8 `--download-cache` was dropped, pip should be using cache by default that can be turned off with `--no-cache-dir` – Ski Jan 31 '16 at 18:06
  • I don't see PIP_DOWNLOAD_CACHE mentioned in https://pip.pypa.io/en/stable/topics/caching/ - but I guess this is where we find out about generic pip env vars: https://pip.pypa.io/en/stable/topics/configuration/#environment-variables – rfay Mar 30 '23 at 20:18
  • RIP pip-accel :(((( – jtlz2 May 15 '23 at 19:58
8

You could 

# download and extract package to build path
pip install --no-install matplotlib

# the build path could be found by 
pip install --help|grep Unpack\ packages\ into -A 2

# then rm pip-delete-this-directory.txt inside the build path
# this prevent pip from removing package from the build directory after install
# you could check the content of the file
rm build/pip-delete-this-directory.txt

# from now on you could install matplotlib quickly
# this uses single build directory 
# and can speed up compiling by caching intermediate objects.
pip install --no-download matplotlib

Also, you could manually download the package

pip install -d dir_for_packages matplotlib

Then install it by un-tar and python setup install later.

The pip install --download-cache works in a similar way w/ extra checking: it firstly search for the latest or specified version of the target package from web, if the search has result and there is cached package in the directory specified by download-cache, the cached package will be used instead of downloading. For example, 

pip install --download-cache . pymongo

will download pymongo package to current directory:

http%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpymongo%2Fpymongo-2.1.1.tar.gz
http%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpymongo%2Fpymongo-2.1.1.tar.gz.content-type
okm
  • 23,575
  • 5
  • 83
  • 90