4

I have a Dockerfile that looks like this:

COPY ./aaa/package.json ./aaa/package.json
COPY ./bbb/package.json ./bbb/package.json
COPY ./ccc/package.json ./ccc/package.json
WORKDIR aaa
RUN npm install
COPY ./aaa ./aaa

Basically module aaa uses bbb and ccc as local npm modules

Is it possible to write it so that the first 3 COPY instructions are done with a single COPY instruction so that they are 1 layer instead of 3? (I realize there's a 4th layer with the last COPY)

I still need the last COPY separate. That's intentional. The reason for splitting out the last layer is that the npm install is only dependent on the package.json files, and this way if I change source code, it doesn't need to rebuild all layers, just the last one. Only if I change the package.json files does it need to rebuild the first layer and do a new npm install. This was a fine pattern for me using a single module, but now that I've got a main module that is using local submodules (local npm modules) I'm stuck on how to reduce the number of COPY instructions to reduce the number of layers. A full description of this technique is documented (and recommended) at nodejs.org in the article "Dockerizing a Node.js web app"

Worth mentioning that it technically works as is, but it's inefficient because it creates extra layers for the extra copies when it seems like it should be possible to somehow get the first three COPY instructions combined to get one layer.

Wyck
  • 10,311
  • 6
  • 39
  • 60
  • theres already a solution in this link https://stackoverflow.com/questions/30256386/how-to-copy-multiple-files-in-one-layer-using-a-dockerfile – Abhishek D K Apr 08 '19 at 04:52
  • 1
    @AbhishekDK can you explain how to use that solution to target different folders though? AFAICT the last parameter is the one-and-only output directory, so you couldn't copy to **both** `./aaa/` **and** to `./bbb/` with that technique. Am I mistaken? – Wyck Apr 08 '19 at 12:36
  • Possible duplicate: [Copy multiple directories with one command](https://stackoverflow.com/q/37715224/596285) – BMitch Apr 08 '19 at 13:15
  • Another duplicate: https://stackoverflow.com/questions/52605123/copy-several-directories-to-another-directory – BMitch Apr 08 '19 at 13:17
  • 1
    @BMitch Nope. That's different too. That question talks about copying the entire directores, `COPY dirone ./dirone` whereas I'm cherry picking only package.json out of each directory. `COPY dirone/package.json ./dirone/package.json`. There's a huge difference. – Wyck Apr 08 '19 at 13:17
  • 1
    @BMitch again, no. Neither of those questions are pulling individual files out of a directory. They are substantially different. – Wyck Apr 08 '19 at 13:18
  • Regardless of whether it's a file or directory, you need to restructure your build context to get it down to a single copy. But going from 3 to 1 layers for single files like this will be unlikely to make any noticeable difference in your resulting image performance. – BMitch Apr 08 '19 at 13:19
  • @BMitch If you can write the answer here with the technique described, I'll accept it. I'm not getting the technique from those other answers becuase they don't explain how to "restructure your build context" to omit the contents of the directory except for the files I want. – Wyck Apr 08 '19 at 13:21
  • I was thinking may a multi-stage build with different .dockerignore files for each stage? So that the COPY in the first stage would only copy the package.json files, but the COPY in the second stage would copy everything else. But I have no idea how. – Wyck Apr 08 '19 at 13:34

1 Answers1

3

You're trying to convert this to a many-to-many copy. This isn't supported by the Dockerfile syntax. You need to have a single destination directory on the right side. And if your source is one or more directories, you need to be aware that docker will copy the contents of those directories, not the directory name itself. The result is that you want:

COPY json-files/ ./

And then you need to organize your build context (in docker build . the . or current directory is your build context that is sent to the docker server to perform the build) with a directory called json-files (could be any name) that contains only the files in the directory structure you want to copy:

| json-files/
|-  aaa/package.json
|-  bbb/package.json
\-  ccc/package.json

Option 2:

You could structure your build as a multi-stage build to get this down to a single layer without modifying your build context itself:

FROM scratch as json-files
COPY ./aaa/package.json /json-files/aaa/package.json
COPY ./bbb/package.json /json-files/bbb/package.json
COPY ./ccc/package.json /json-files/ccc/package.json

FROM your_base
COPY --from=json-files /json-files .
WORKDIR aaa
RUN npm install
COPY ./aaa ./aaa

This second option is the same as the first from the the view of your COPY command, it just has an image as it's context instead of the build context sent over with the build command.

All this said, changing from 3 copy commands to 1, for small individual files that do not overwrite each other, is unlikely to have any noticeable impact on your performance and this looks like premature optimization.

BMitch
  • 231,797
  • 42
  • 475
  • 450