563

In a Dockerfile, I have

COPY . .

I want to exclude an entire directory, in my case, node_modules directory.

Something like this:

   COPY [all but **/node_modules/**] .

Is this possible with Docker?

Alexander Mills
  • 90,741
  • 139
  • 482
  • 817
  • 69
    How about [`.dockerignore`](https://docs.docker.com/engine/reference/builder/#dockerignore-file) ? – anthony sottile May 02 '17 at 21:50
  • I am looking for answer to this related question: https://stackoverflow.com/questions/50547489/specify-dockerignore-from-command-line – Alexander Mills May 26 '18 at 22:34
  • 3
    @AnthonySottile dockerignore prevents files being part of the build context. – Marcello Romani Nov 26 '21 at 15:55
  • 1
    i have .venv directory in the .dockerignore (.venv and **/.venv) and it still gets copied with COPY . . in dockerfile – ierdna Jul 21 '22 at 23:16
  • 2
    My project has 50 directories, and takes around 20 minutes to compile. Part of the build script also includes other minor compilations at the end (executed by other scripts), which currently fail. I essentially only want to separate these two compilation processes and debug those extra scripts. Without writing 50+ COPY statements, specifying everything but these scripts. The key point here is wanting to cache the result of the first compilation. – Beolap Nov 07 '22 at 02:29

7 Answers7

864

Create file .dockerignore in your docker build context directory (so in this case, most likely a directory that is a parent to node_modules) with one line in it: 

**/node_modules

although you probably just want:

node_modules

Info about dockerignore: https://docs.docker.com/engine/reference/builder/#dockerignore-file

Soviut
  • 88,194
  • 49
  • 192
  • 260
vith
  • 8,916
  • 1
  • 12
  • 4
  • 16
    thanks, so the "COPY . ." command does not need to change, I guess – Alexander Mills May 03 '17 at 17:27
  • Not really: ```/config/nginx/ /config/logstash/```, i find this in the container bash: ```ls config/ gunicorn logs logstash nginx``` – SalahAdDin Feb 06 '18 at 23:55
  • 3
    @SalahAdDin for the same reason you didn't write `ls /config/`, did you try adding those paths to your .dockerignore without the leading `/` ? – Pablo Feb 24 '18 at 06:21
  • I am looking for answer to this related question: https://stackoverflow.com/questions/50547489/specify-dockerignore-from-command-line – Alexander Mills May 26 '18 at 22:34
  • 20
    Just `node_modules` does not work in subdirectories. – rustyx Jul 19 '18 at 09:07
  • 97
    This doesn't really solve the problem if you want to have it COPY'd but just not in that layer... – Wes Nov 24 '19 at 19:15
  • 3
    This answer is fundamentally wrong. .dockerignore prevents files being sent to the Docker server, thus excludes them from the build context. What would be useful instead is the ability to copy a folder excluding some of the content of that folder. – Marcello Romani Nov 26 '21 at 15:53
  • 1
    .dockerignore is completely unrelated to COPY – Peter Kionga-Kamau Apr 10 '22 at 08:51
  • 5
    @PeterKionga-Kamau If that's the case, could you explain why the `.dockerignore` docs say "This helps to avoid unnecessarily sending large or sensitive files and directories to the daemon and potentially adding them to images using ADD or COPY." in the first paragraph? – ceejayoz May 15 '22 at 13:08
  • 2
    @ceejayoz It would not be the first time that the docker documentation was wrong. `.dockerignore` does prevent files from being sent to the **daemon** but if you use COPY from a local folder it will copy everything in that folder whether or not you have an entry in the `.dockerignore` file. You can easily verify this by running a simple build – Peter Kionga-Kamau May 16 '22 at 19:37
  • Does it work for nested folders? Eg having .dockerignore at app level (for a monorepo) – Eric Burel Sep 22 '22 at 07:51
142

For those who can't use a .dockerignore file (e.g. if you need the file in one COPY but not another):

Yes, but you need multiple COPY instructions. Specifically, you need a COPY for each letter in the filename you wish to exclude.

COPY [^n]*    # All files that don't start with 'n'
COPY n[^o]*   # All files that start with 'n', but not 'no'
COPY no[^d]*  # All files that start with 'no', but not 'nod'

Continuing until you have the full file name, or just the prefix you're reasonably sure won't have any other files.

Alex
  • 1,537
  • 1
  • 7
  • 5
  • 5
    This won't wort because you wil get a `COPY failed: no source files were specified` when there are no files starting with 'no', but not 'nod'. – mheck Apr 08 '19 at 13:02
  • 2
    Alternatively, you could use two different sub-directories (for different copy targets), so that e.g. in my nginx setup, I would have `src/` will go to `/var/www` and `/conf/nginx.conf` go to `/etc/nginx/conf.d/default.conf`. Also, you could copy everything, and `RUN rm -r conf` later (too keep the source tree as it was). I would like to hear if someone has another suggestion. – Yuval Feb 13 '20 at 10:11
  • 2
    What would be a solution for specific file types or file endings if I would do it like this? – Lea Reimann Feb 19 '20 at 13:50
  • 15
    The only downside of this approach is that your folder structure is flattened – Islam Mar 07 '20 at 15:38
  • 1
    If you have only few directories to be copied (say only '/app' or '/src'). A simpler way would be to copy directories individually (to keep their structure) and then you only need one `COPY [^node_modules]*` to copy all top level files. – smdufb Jun 25 '20 at 10:49
12

FOR A ONE LINER SOLUTION, type the following in Command prompt or Terminal at project root.

echo node_modules >> .dockerignore

This command appends "node_modules" in the .dockerignore file. If the .dockerignore does not exist already, it will create a new one. Replace node_modules with the folder you want to exclude.

Warning: If you are new to Docker ecosystem and/or you already have the .dockerignore file in your project, please take a backup before proceeding.

BONUS: (as pointed out by Joey Baruch)

(To CREATE/OVERWRITE the .dockerignore file via PowerShell, which can be handled by Docker):
>> echo node_modules | Out-File -Encoding UTF8 .dockerignore
Naveen Kumar V
  • 2,559
  • 2
  • 29
  • 43
  • 3
    Don't do this in PowerShell, it outputs UTF-16 by default, and I don't think docker can handle that – Joey Baruch Oct 02 '20 at 18:37
  • 17
    Do you really want people to lost their .dockerignore contents ? :) echo node_modules >> .dockerignore if so :) – webcitron Jan 09 '21 at 12:17
  • 4
    I don't know why this answer has so many upvotes, when it's dangerous if people just copy-paste it in a project that already has a .dockerignore. Also, without explaining what's the functionality of the .dockerignore file, someone could think that this is a way of saying Docker to don't use that folder in the next build. Take into account you answer to people who doesn't usually know how Docker works. – Daniel Campos Olivares Mar 11 '21 at 08:07
  • webcitron and DanielCampos, I edited my answer. Thanks. Hope that helps. :) – Naveen Kumar V Mar 17 '21 at 05:56
  • 1
    @JoeyBaruch Thanks for mentioning it. I updated my answer for the PowerShell as well. :) – Naveen Kumar V Mar 18 '21 at 13:34
  • 1
    This is completely unrelated to the COPY command. `.dockerignore` is NOT going to stop those files from being copied, it only affects the build context. Why do these answers keep getting upvotes? – Peter Kionga-Kamau Apr 10 '22 at 08:56
  • @PeterKionga-Kamau I dont think so. Kindly refer https://www.docker.com/blog/keep-nodejs-rockin-in-docker/ where they suggest to keep node_modules in the .dockerignore. Kindly advice if I am wrong. – Naveen Kumar V May 15 '23 at 14:52
  • @DanielCamposOlivares I am just providing a way to do it. I can't teach him completely on the docker(ignore) ecosystem. It could have actually helped the OP if I was right. Danke Schoen :) – Naveen Kumar V May 15 '23 at 14:58
  • 1
    @NaveenKumarV a way to do it, which doesn't align with the question. The OP explicitly asked how to prevent a concrete directory to be copied during a concrete COPY instruction, if you add it to the `.dockerignore`, you're not only preventing the directory to be copied within that instruction, but on the entire build, since you're excluding it from the build context. Anyway, good that the answer improved, because when I answered here, it was REALLY dangerous (by using the redirect `>` operator) and without any context at all. – Daniel Campos Olivares May 16 '23 at 10:18
  • @DanielCamposOlivares Thanks for your insights... which made me to improvise my answer. :) – Naveen Kumar V May 16 '23 at 11:10
  • 1
    @NaveenKumarV Correction: if you use volume mapping, it will map everything in spite of anything you put the .dockerignore file. If you don't use volume mapping, then you can use .dockerignore to keep host files out of the image. – Peter Kionga-Kamau Jun 19 '23 at 19:53
11

Excluding node_modules from current directory 

node_modules

Excluding node_modules in any immediate subdirectories

*/node_modules

Here is the official docs

kta
  • 19,412
  • 7
  • 65
  • 47
  • 11
    You can do both with `**/node_modules` – bfontaine Sep 17 '21 at 10:13
  • 1
    @PeterKionga-Kamau They do have an effect. The ADD and COPY command are explicitly mentioned as being affected by dockerignore: https://docs.docker.com/engine/reference/builder/#dockerignore-file – Leon Lucardie Jul 26 '22 at 13:40
  • @LeonLucardie did you try or are you relying on the documentation being accurate? – Peter Kionga-Kamau Aug 02 '22 at 20:14
  • 2
    @PeterKionga-Kamau Tried multiple times. Today I've re-verified with Docker v20.10.16 and any files that match .dockerignore entries are not picked up by COPY or ADD commands I've added to Dockerfile. – Leon Lucardie Aug 04 '22 at 12:15
  • @LeonLucardie Perhaps they have issued a fix. As of May 16, 2022, .dockerignore did prevent files from being sent to the *daemon* but if you used COPY from a local folder it copied everything in that folder whether or not you had an entry in the .dockerignore file, and I verified this by running a simple build – Peter Kionga-Kamau Aug 08 '22 at 03:51
  • 2
    @PeterKionga-Kamau v20.10.16 was the most recent version as of May 16. I've just retried once more with a fresh dockerfile and all patterns/files I have in my .dockerignore are not picked up by to the COPY or ADD command. Do you have an example of when it did not work? – Leon Lucardie Aug 08 '22 at 07:54
  • @LeonLucardie Maybe, but I am not sure where I saved it. It could take a while to find. I'll get to it at some point though. Do you mind sharing the one you have that's working the way you expect? – Peter Kionga-Kamau Aug 09 '22 at 03:51
  • 2
    @PeterKionga-Kamau Here you go: https://github.com/leonluc-dev/dockerignoreexample After building and running the docker image, inspecting it should show that the entry in the dockerignore file (testfile1.txt in this case) was not copied to the docker image. – Leon Lucardie Aug 17 '22 at 09:43
6

For those using gcloud build:

gcloud build ignores .dockerignore and looks instead for .gcloudignore

Use:

cp .dockerignore .gcloudignore

Source

nik7
  • 806
  • 3
  • 12
  • 20
JesusIniesta
  • 10,412
  • 1
  • 35
  • 28
  • are the file formats the same between the two? – edjm Oct 20 '22 at 16:51
  • 1
    _"The syntax of .gcloudignore borrows heavily from that of .gitignore; see https://git-scm.com/docs/gitignore or man gitignore for a full reference"_ - https://cloud.google.com/sdk/gcloud/reference/topic/gcloudignore @edjm – JesusIniesta Oct 25 '22 at 10:14
4

Adding .dockerignore works for me. One additional point Those who are trying this solution on Windows , windows will not let you create .dockerignore file (as it doesn't by default allows creating file starting with .)

To create such file starting with . on Windows, include an ending dot also, like : .dockerignore. and hit enter ( provided you have enabled view extension options from folder options )

Paras Patidar
  • 1,024
  • 11
  • 11
4

I used a multi stage build approach since I needed one stage to have access to the file but not another stage so .dockerignore wouldn't work:

FROM ruby AS builder

COPY app/ app/

# Do stuff with app

# remove the stuff you don't want
RUN rm -Rf app/assets

FROM ruby AS publish

# In my real version I needed the absolute path to builder WORKDIR.
# Since I'm copying from the builder stage, app/assets won't exist
# and neither will it be part of the publish image.
COPY --from=builder app app
moger777
  • 1,117
  • 11
  • 16
  • 4
    this is wrong. app/assets is still in the image - though in a layer not visible. – Andreas Wittig Oct 08 '22 at 13:46
  • 1
    @AndreasWittig, in a multi stage build only what you copy in the final stage is kept. In my example we have a builder stage which removes some files. Those files would still exist in a layer on the builder image but as you say, are no longer part of that layer. When we copy into the publish stage, since those files aren't visible they won't be included in the image from the publish stage. – moger777 Oct 11 '22 at 20:31
  • I think this is the cleanest way for now. Lots of people just don't want to maintain two ignore files. (gitignore, dockerignore). – agate Feb 14 '23 at 18:28
  • It's only clean with a multi stage build. Unless that, @AndreasWittig is right: the ignored files are still in a layer of the image. – Paul Rey Apr 26 '23 at 21:15
  • But my example is a multi stage build. There is the builder stage and the publish stage. The idea is the publish image would be pushed to deployment. – moger777 May 05 '23 at 14:19
  • Doesn't the intermediate layer still contain those files - and so if there are many of them you get a whole load of image bloat, even with the rm afterwards? – jtlz2 Jun 08 '23 at 07:52
  • 1
    @jtlz2, according to https://docs.docker.com/build/building/multi-stage/, using a multistage build allows you to copy what you want from another stage and it discards the rest. There are two images built, the builder image has all the bloat but the publish images, which is the one you would deploy wouldn't have it since you are only copying what wasn't removed. – moger777 Jun 08 '23 at 18:46
  • Amazing, thanks for the link! – jtlz2 Jun 08 '23 at 19:37