Docker compose: Multiple isolated environments on a single host with overridable volumes and ports

Question

We have a requirement of creating multiple isolated envs of our app in a single host, using a single compose file.

I realized that by specifying the project name using -p option we can create multiple isolated envs using docker compose in a single host.

However is it possible to override the ports: and volumes: in the compose file for different environment, without having 2 seperate docker compose files?

For instance, I would like to override the following properties, preferably through command-line args.

For dev environment

ports:
 8081:8080

volumes:
 /etc/myapp/dev/properties/:/etc/myapp/properties

For QA environment

ports:
 8082:8080

volumes:
 /etc/myapp/qa/properties/:/etc/myapp/properties

I think that you can use env variables for it. See more here http://stackoverflow.com/questions/29377853/how-to-use-environment-variables-in-docker-compose — Cortwave, Apr 06 '16 at 15:06

bsferreira · Accepted Answer · 2016-04-06T22:52:12.903

You can use a template.yml and pass the variables you want to generate docker-compose.yml

First, create a template.yml with the following content:

version: "2"
...
ports:
 "$HOST_PORT":8080

volumes:
 "$HOST_VOLUME":/etc/myapp/properties

Now, you can create a script with the variables you want by environment. For dev environment it would look like this:

#!/bin/bash

# Variables to use in template.yml
export HOST_PORT="8081"
export HOST_VOLUME="/etc/myapp/dev/properties/"

# build docker-compose.yml from the template
source env.sh; rm -rf docker-compose.yml; envsubst < "template.yml" > "docker-compose.yml";

This will generate a docker-compose.yml with the concrete values.

Here's an usage example: https://github.com/bsferreira/mysql-fabric

score 1 · Answer 2 · answered Aug 05 '21 at 19:27

Needed this as well and stumbled upon this question. There is built in support for isolating environments:

Multiple isolated environments on a single host

Compose uses a project name to isolate environments from each other. You can make use of this project name in several different contexts:

    * on a dev host, to create multiple copies of a single environment, such as when you want to run a stable copy for each feature branch of a project
    * on a CI server, to keep builds from interfering with each other, you can set the project name to a unique build number
    * on a shared host or dev host, to prevent different projects, which may use the same service names, from interfering with each other

The default project name is the basename of the project directory. You can set a custom project name by using the -p command line option or the COMPOSE_PROJECT_NAME environment variable.

The default project directory is the base directory of the Compose file. A custom value for it can be defined with the --project-directory command line option.

https://docs.docker.com/compose/#multiple-isolated-environments-on-a-single-host

score 1 · Answer 3 · answered Nov 18 '21 at 06:14

You can use Variable substitution with Declare default environment variables in file.

For example, my compose project structure is

C:\docker-compose
└───multiple-envs
    │   .env
    │   .env-dev
    │   docker-compose.yml
    │
    ├───dev-files
    └───files

docker-compose.yml, file content below, you can set ports and other values using ${ENV_VAR} syntax. It will automatically substitute by docker compose cli.

You can instruct docker compose to show error message when the ${ENV_VAR} not set or empty using syntax ${HOST_MYSQL_PORT:?HOST_MYSQL_PORT is not set}, after :? is the error message.

services:
  mysqldb:
    image: mysql
    restart: always
    ports:
      - ${HOST_MYSQL_PORT:?HOST_MYSQL_PORT is not set}:3306
    volumes:
      - type: bind
        source: ${VOLUMES_SOURCE:?VOLUMES_SOURCE is not set}
        target: /mnt
        read_only: true
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:?MYSQL_ROOT_PASSWORD is not set}
      MYSQL_USER: ${MYSQL_USER:?MYSQL_USER is not set}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD:?MYSQL_PASSWORD is not set}

.env file, set ENV_VAR=VAL in file called .env, let's say for production.

HOST_MYSQL_PORT=13306
VOLUMES_SOURCE=./files
MYSQL_ROOT_PASSWORD=p419460507
MYSQL_USER=u130085860
MYSQL_PASSWORD=p689273542

.env-dev file, set ENV_VAR=VAL in file called .env-dev, let's say for development.

HOST_MYSQL_PORT=23306
VOLUMES_SOURCE=./dev-files
MYSQL_ROOT_PASSWORD=dev419460507
MYSQL_USER=dev130085860
MYSQL_PASSWORD=dev689273542

To compose in different config, specify environment file when invoke docker compose cli, --env-file multiple-envs\.env for production or --env-file multiple-envs\.env-dev for development.

C:\docker-compose> docker-compose --project-directory multiple-envs --env-file multiple-envs\.env up --detach
[+] Running 2/2
 - Network multiple-envs_default      Created                                                                                                                                                     0.6s  
 - Container multiple-envs-mysqldb-1  Started                                                                                                                                                     1.8s

C:\docker-compose> docker-compose --project-directory multiple-envs --env-file multiple-envs\.env-dev up --detach
[+] Running 1/1
 - Container multiple-envs-mysqldb-1  Started                                                                                                                                                    11.7s

Docker compose: Multiple isolated environments on a single host with overridable volumes and ports

3 Answers3