2

I have a repository where I need to create several Dockerfiles, but each of them should have a different context.

I like the solution posted here, but it doesn't fully fit with my use case.

NO, THIS IS NOT A DUPLICATE. IT'S A DIFFERENT USE CASE. PLEASE KEEP READING.

I know it's better to exclude from the context unnecessary folders, especially if they are big. Well, my project consists of several folders, some of them are really huge.

For simplicity, suppose this is the file tree of my project:

  • hugeFolder1/
  • hugeFolder2/
  • littleFolder1/
  • littleFolder2/

And suppose that I need to create two Dockerfiles (following the solution that I previously mentioned):

  • docker/A/Dockerfile <- let's call this Dockerfile "A"
  • docker/B/Dockerfile <- let's call this Dockerfile "B"
  • docker-compose.yml

Now the point is:

  • A only needs hugeFolder1 and both the little folders.
  • B only needs hugeFolder2 and both the little folders.

So I would like to exclude the unneeded huge folders respectively.

What is the best way to achieve this?

Francesco Borzi
  • 56,083
  • 47
  • 179
  • 252

1 Answers1

1

Edit: Previous answer was adding folders to the image that were outside the build context, which won't work. Additionally OP clarified the contents and how the image will be used in the comments, showing a very good use case for multi stage builds.

I'll take a stab at it, based on the info provided.

Firstly, you can't exclude folders from a given docker context. If you use docker build -t bluescores/myimage /some/path, your context is /some/path/**/*. There's no excluding the huge folders, or the little folders, or anything in them.

Second, in order to use ADD or COPY to bring files into your docker image, they must exist in the build context.

That said, it sounds like you'll end up using various combinations of the huge and little folders. I think you'll be better off sticking with your existing strategy you've outlined, with some optimizations - namely using multi stage builds.

Skip docker-compose for now

The solution here that you reference isn't really aiming to solve the problem of closely controlling context. It's a good answer to a totally different question than what you're asking. You are just trying to build the images right now, and while docker-compose can handle that, it doesn't bring anything to the table you don't have with docker build. When you need to orchestrate these containers you're building, then docker-compose will be incredible.

If you aren't sure you need docker-compose, try doing this without it. You can always bring it back into the mix later.

Now my image is gigantic

See if multi-stage builds are something you can make use of.

This would essentially let you cherry pick the build output from the image you COPY the huge folders into, and put that output in a new, smaller, cleaner image.

bluescores
  • 4,437
  • 1
  • 20
  • 34
  • thank you very much for your answer. `hugeFolder1` would be full of SQL files that I need to combine and import to my database (then I could remove them). `hugeFolder2` is full of source files that I need to compile (then I could remove them). With this information, are there any optimisations possible? – Francesco Borzi Jan 07 '19 at 21:04
  • 1
    Regarding image size (which sounds like it will be pretty big because of these huge folders), multi stage builds would be the way to go. Keep in mind with multi stage, you aren't *removing* anything from the final image, you're plucking some build artifacts - like a compiled project - out of one container and leaving all those huge folders behind. Sounds like that will be a good option to explore here. Spend time just getting it working without multi stage, then read up and try it. – bluescores Jan 07 '19 at 21:07
  • `ADD failed: Forbidden path outside the build context: ../../hugeFolder1 ()` – Francesco Borzi Jan 08 '19 at 22:43
  • 1
    Wow, massive oversight on my part. This solution I've posted won't work at all because the huge folders aren't included in the build context. You could still use multistage builds to greatly optimize the final image, regardless. But you're going to have to include that root directory with the huge and little directories in the build image context. – bluescores Jan 09 '19 at 17:02
  • @FrancescoBorzì I updated my answer to reflect the comments here. Removed the incorrect bit about using `ADD` outside the build context. Overall I think your current strategy as you've laid out is fine, but you can greatly shrink the final images using multi stage as we've been talking about. – bluescores Jan 12 '19 at 13:07
  • thanks @bluescores, I've solved using multi stage builds. Really helpful. – Francesco Borzi Jan 12 '19 at 14:49
  • I was wondering how I can use multi stage builds in Docker Compose: https://stackoverflow.com/questions/54207856/docker-compose-and-external-images-multi-stage-builds – Francesco Borzi Jan 15 '19 at 22:41