How to organize multiple Dockerfiles, docker-compose.yaml and .dockerignore

Question

After reviewing the answers on How to name Dockerfiles I'm still left with a lot of questions.

In my case, I am needing at least two Dockerfiles, a docker-compose.yaml and a .dockerignore. It sounds like using an extension such as <purpose>.Dockerfile or Dockerfile.<purpose> has the drawback of losing the functionality of using autobuilder at hub.docker.com.

So others are suggesting you keep each Dockerfile in a directory and build from there.

So maybe something like:

dockerfiles/
--python_consumer/
-----Dockerfile
--python_producer/
-----Dockerfile
--docker-compose.yaml
--.dockerignore

Would the .dockerignore in this case work globally for all of the dockerfiles? Any big drawbacks to structuring this way?

Example of my docker-compose.yaml without separate directories and one combined consumer/production image for context.

version: '3.8'
services:
  standalone:
    hostname: standalone
    container_name: standalone
    image: apachepulsar/pulsar:2.8.1
    ports:
      - 8080:8080 # exposed would be a better practice
      - 6650:6650 # exposed would be a better practice
    command: bin/pulsar standalone
    healthcheck:
      test: ["CMD", "nc", "-vz", "localhost", "6650"]
      interval: 20s
      timeout: 5s
      retries: 5
    networks:
      - conprod
  conprod:
    hostname: conprod
    container_name: conprod
    build:
      context: .
      dockerfile: ./Dockerfile
    restart: on-failure # best practice is probably "unless-stopped"
    depends_on:
      - standalone
    networks:
      - conprod
networks:
  conprod:

What does the `docker-compose.yml` look like; are the subdirectories usable as self-contained build contexts? If so then you should put `.dockerignore` next to the `Dockerfile`s. — David Maze, Oct 13 '21 at 20:39
I've added my `docker-compose.yaml` for context. Currently my `Dockerfile` is in the root directory with the `.dockerignore` and `docker-compose.yaml`. But I'm going to need two `Dockerfile`s to build out two containers for two different python applications. — Hofbr, Oct 13 '21 at 20:53
The [`.dockerignore` file](https://docs.docker.com/engine/reference/builder/#dockerignore-file) gets read from the `build: { context: }` directory; it won't have an effect anywhere else. — David Maze, Oct 13 '21 at 21:29
So to edit the above, my `context:` would be changed to `./dockerfiles`? — Hofbr, Oct 13 '21 at 21:56

score 9 · Accepted Answer · answered Oct 13 '21 at 23:25

When you build an image, you send the Docker daemon a build context; in your Compose setup this is the directory named in the build: { context: } setting. The .dockerignore file must be in that exact directory and nowhere else. Its actual effect is to cause files to be excluded from the build context, which can result in a faster build sequence.

The build context's other important effect is that all Dockerfile COPY directives are considered relative to that directory; you cannot COPY from parent or sibling directories. So if files are shared between projects, you must set the context directory to some ancestor directory of all of the files that will be included, and COPY directives will be relative to that directory (even if the Dockerfiles are in per-project directories). See also How to include files outside of Docker's build context?

If your projects are completely separate: maybe there's a front-end and a back-end project, or in your case a producer and a consumer that share a message format but not any actual code. Then in this case:

Put a Dockerfile, named exactly Dockerfile, in each project subdirectory
Put a .dockerignore file in each project subdirectory (it cannot be in the parent directory)
COPY directives are relative to the project subdirectory
```
COPY requirements.txt ./
```

In the Compose file, you can use the shorthand build: directory syntax, since you have the standard (default) dockerfile: name

version: '3.8'
services:
   producer:
     build: ./python_producer
     environment:
       - RABBITMQ_HOST=rabbitmq
   consumer:
     build: ./python_consumer
     environment:
       - RABBITMQ_HOST=rabbitmq
   rabbitmq:
     image: rabbitmq:3
     hostname: rabbitmq # RabbitMQ is very unusual in needing to set this

If your projects share code or other files: in your example maybe you define Python data structures for the message format in shared code. This in this case:

Put a Dockerfile, named exactly Dockerfile, in each project subdirectory
Put a single .dockerignore file in the project root
COPY directives are relative to the project root directory
```
COPY python_producer/requirements.txt ./
```

In the Compose file you need to specify context: . and dockerfile: pointing at a per-component Dockerfile

version: '3.8'
services:
   producer:
     build:
       context: .
       dockerfile: python_producer/Dockerfile
     environment:
       - RABBITMQ_HOST=rabbitmq
   consumer:
     build:
       context: .
       dockerfile: python_consumer/Dockerfile
     environment:
       - RABBITMQ_HOST=rabbitmq
   rabbitmq:
     image: rabbitmq:3
     hostname: rabbitmq # RabbitMQ is very unusual in needing to set this

Nice answer, David! I just would like to recommend using `./` in the `dockerfile:` properties too. It can prevent misinterpretation of the name folder. — Juan Fontes, Oct 14 '21 at 03:43
to be clear @JuanFontes you're recommending './python_producer/Dockerfile' and './python_consumer/Dockerfile' ? — Hofbr, Oct 14 '21 at 18:11

How to organize multiple Dockerfiles, docker-compose.yaml and .dockerignore

1 Answers1