What is pip's `--no-cache-dir` good for?

Question

I've recently seen the --no-cache-dir being used in a Docker file. I've never seen that flag before and the help is not explaining it:

 --no-cache-dir              Disable the cache.

Question: What is cached?
Question: What is the cache used for?
Question: Why would I want to disable it?

Documentation is pretty extensive on that one: https://pip.pypa.io/en/stable/reference/pip_install/#caching — emredjan, Aug 09 '17 at 15:24
the first google result gives a pretty good description of it's usage https://pip.pypa.io/en/stable/reference/pip_install/ — mikea, Aug 09 '17 at 15:24
When you are building a docker image you want it to be light. This helps avoiding bloating the image with cached files. — Victor Lamoine, Jun 11 '18 at 18:17
@mikea Ironically, for `pip no-cache-dir`, this is now the first result. — hlongmore, Oct 01 '18 at 21:08
Might be helpful to add a `docker` tag, since Docker is referenced by the question and most of the answers. — Jacob Wan, Feb 12 '23 at 16:37

score 256 · Accepted Answer · edited Jul 01 '20 at 17:29

256

Cached is: store away in hiding or for future use
Used for

store the installation files(.whl, etc) of the modules that you install through pip
store the source files (.tar.gz, etc) to avoid re-download when not expired

Possible Reason you might want to disable cache:

you don't have space on your hard drive
previously run pip install with unexpected settings
- eg:
  - previously run export PYCURL_SSL_LIBRARY=nss and pip install pycurl
  - want new run export PYCURL_SSL_LIBRARY=openssl and pip install pycurl --compile --no-cache-dir
you want to keep a Docker image as small as possible

Links to documentation

https://pip.pypa.io/en/stable/reference/pip_install/#caching – @emredjan https://pip.pypa.io/en/stable/reference/pip_install/ - @mikea

edited Jul 01 '20 at 17:29

GG.

21,083
14
84
130

answered Aug 09 '17 at 15:25

Stack

4,116
3
18
23

5

Why would I want to store the installation files? – Martin Thoma Aug 09 '17 at 15:31
16

To avoid downloading again and again. Suppose, you uninstall a module, when you install it next time, it will use the files from the cached dir – Stack Aug 09 '17 at 15:33
21

Ah, ok. So for a docker image which is only used for deployment (hence no "manual" actions") there is no reason not to use `--no-cache-dir`, correct? – Martin Thoma Aug 09 '17 at 15:58
Could these links be turned into html links? – Hatshepsut May 11 '18 at 19:18
@Hatshepsut Done! – Stack May 12 '18 at 10:21
@ozgur: I think you might have misread my comment. I argue as well that you should use the flag – Martin Thoma Jul 18 '18 at 15:11
21

Yes @MartinThoma, on a production docker image, you want to use pip --no-cache-dir, because no one else will install any packages, and storage space is even more valuable... – Ozgur Ozturk Jul 19 '18 at 15:50
12

Haha I don't think the OP meant "what does 'cached' mean?" for question 1. – Arel Jan 15 '19 at 19:09
If you followed the bad practice of updating your development code without incrementing the version, it would use the cache rather than your new file. – boris Feb 26 '20 at 18:00
another reason why you would use `--no-cache-dir` https://intellipaat.com/community/9420/removing-pips-cache – enjoy343322434 Feb 04 '21 at 09:59
Many of the links here have been moved in the docs. Latest as of 2021-09-16 is https://pip.pypa.io/en/stable/topics/caching/. – Jeff Wright Sep 16 '21 at 13:40

Philip Tzou · Answer 2 · 2019-09-25T22:21:16.567

167

I think there is a good reason to use --no-cache-dir when you are building Docker images. The cache is usually useless in a Docker image, and you can definitely shrink the image size by disabling the cache.

edited Sep 25 '19 at 22:21

answered Aug 14 '19 at 07:58

Philip Tzou

5,926
2
18
27

38

you can use [`ENV PIP_NO_CACHE_DIR=1`](https://stackoverflow.com/a/60270281/2248627) in docker for python 3.6.10 and higher images – Levon Feb 19 '20 at 00:36
Fwiw, with pip 18.1 you need to [set a falsy value](https://pip.pypa.io/en/stable/user_guide/#:~:text=falsy%20values) for that env var to work. For anything else I get a `TypeError: expected str, bytes or os.PathLike object, not int` since the value gets parsed into a `posixpath`. – dtk Feb 22 '21 at 11:48
1

`RUN pip install --no-cache-dir -r requirements.txt` for docker images – Timothy L.J. Stewart Apr 23 '22 at 22:02
Is this still important in **multi-stage builds**? – Alex Peters Jun 05 '22 at 12:16
@AlexPeters Depends on how you make the multi-stage builds. I found the cleanest way is to install packages in builder image and copy the whole `site-packages/` to final image. This way you don't need to use any extra parameters of `pip` since the cache are not copied. – Philip Tzou Jun 06 '22 at 16:08

score 15 · Answer 3 · answered Mar 07 '19 at 22:36

Another reason to disable the pip cache - if you run pip as a user that does not yet exist, their home directory will be created, but owned by root.

This happens to us when building Amazon AMIs in a chroot - pip is being run as a user that exists on the builder machine, but not in the chroot jail where the AMI is being constructed. This is problematic as that specific user can now not ssh to what was just built as their .ssh directory is not readable by them.

I can't think of any other reason pip would be run as a user that doesn't exist though, so it's very much an edge case.

score 9 · Answer 4 · answered Oct 10 '19 at 11:36

9

Reduce your docker image size if you're having python dependencies in your DockerFile, as your private registries/artifactories or your deployment servcies may have size limitation.

answered Oct 10 '19 at 11:36

Zoe The Paranoid

472
5
11

score 2 · Answer 5 · answered Feb 17 '20 at 10:32

I get permission error for installation of some pip packages if I don't use --no-cache-dir option.

Building wheels for collected packages: pyyaml, bottleneck, nvidia-ml-py3
  WARNING: Building wheel for pyyaml failed: [Errno 13] Permission denied: '/home/user/.cache/pip/wheels/b1'
  WARNING: Building wheel for bottleneck failed: [Errno 13] Permission denied: '/home/user/.cache/pip/wheels/92'
  WARNING: Building wheel for nvidia-ml-py3 failed: [Errno 13] Permission denied: '/home/user/.cache/pip/wheels/7f'

chown /.cache folder didn't help for some reason but with --no-cache-dir it works ok.

This doesn’t answer the question. – bfontaine Aug 09 '23 at 15:05 — bfontaine, Aug 09 '23 at 15:05

score 1 · Answer 6 · edited Aug 09 '23 at 15:04

1

From fastapi official doc

The --no-cache-dir option tells pip to not save the downloaded packages locally, as that is only if pip was going to be run again to install the same packages, but that's not the case when working with containers.

Basically, there is no need to store whatever package cache you're installing locally since it is not required by docker containers.

edited Aug 09 '23 at 15:04

bfontaine

18,169
13
73
107

answered Jan 27 '23 at 16:16

Arnold Ighiwiyisi

31
1
4

What is pip's `--no-cache-dir` good for?

6 Answers6

Linked