Using environment variable for volume name in docker compose

Question

I am using docker compose version 3.3 and want to use environment variable to define the volume name. I looked at a related question, but that seems to be quite old. With long syntax supported in 3.2, is there a way to achieve that? Here is what I tried in my docker compose file:

version: '3.3'
services:
  target:
    image: "my-registry/my-image:${IMAGE_TAG}"
    volumes:
        - type: volume
          source: ${VOLUME_NAME}
          target: /data
    ports:
     - "${TOMCAT_PORT}:8080"

volumes:
  ${VOLUME_NAME}:

Obviously this syntax does not work as volume name is not substituted in the keys and throws the following error:

volumes value Additional properties are not allowed ('${VOLUME_NAME}' was unexpected)

Any help will be much appreciated.

Answer 1

This is expected behavior - Compose only does variable interpolation in values, not keys. See here.

In my project I use external structure:

version: '3.1'
services:
### Code from branch develop ###
  applications:
    image: registry.gitlab.lc:5000/develop/ed/develop.sources:latest
    volumes:
      - developcode:/var/www/develop
    deploy:
      replicas: 1
      update_config:
        parallelism: 1
        delay: 5s
      restart_policy:
        condition: on-failure
      placement:
        constraints: [node.role == manager]
### PHP-FPM ###
  php-fpm:
    image: registry.gitlab.lc:5000/develop/ed/php-fpm-ed-sq:latest
    volumes:
      - developcode:/var/www/develop
    expose:
      - "9000"
    deploy:
      replicas: 2
      update_config:
        parallelism: 1
        delay: 5s
      restart_policy:
        condition: on-failure
      placement:
        constraints: [node.role == manager]
    logging:
      driver: gelf
      options:
        gelf-address: "udp://${GRAYLOG_ADDR}:12201"
        tag: "php-fpm"
### Nginx ###
  nginx:
    image: registry.gitlab.lc:5000/develop/ed/nginx-ed-sq:staging
    volumes:
      - developcode:/var/www/develop
    ports:
      - "80:80"
      - "443:443"
    deploy:
      replicas: 2
      update_config:
        parallelism: 1
        delay: 5s
      restart_policy:
        condition: on-failure
      placement:
        constraints: [node.role == manager]
### Volumes Setup ###
volumes:
  developcode:
    external:
      name: code-${VER}

but first of all I need create external volume manually, e. g.:

export VER=1.1 && docker volume create --name code-$VER

You can see created volume:

docker volume ls
DRIVER              VOLUME NAME
local               code-1.0
local               code-1.1

And after that, deploy services using:

env $(cat .env | grep ^[A-Z] | xargs) docker stack deploy --with-registry-auth --compose-file docker-compose.yml MY_STACK

Answer 2

If you want to avoid conflicts between different projects, you just need to specify --project-name "your_project_name" for docker-compose. It adds namespace for all services and volumes.

For example, you may use the same docker-compose.yml for few projects:

volumes:
    dbdata_mysql:

services:
    mysql_db:
        image: mysql:5.7
        volumes:
            - "dbdata_mysql:/var/lib/mysql"

If you start your projects with:

docker-compose --project-name "first_project" up -d
docker-compose --project-name "second_project" up -d

it will create namespaced volumes:

$ docker volume ls | grep dbdata_mysql

local     first_project_dbdata_mysql
local     second_project_dbdata_mysql

Answer 3

This is how I got it done for my case inside .env

# To use a code volume create the volume mapping an existing folder like this and pass the value of <volume_name> below
# docker volume create --driver local --opt type=none --opt device=<full_path_to_folder> --opt o=bind <volume_name>
PROJECT_VOLUME_CODE=myproject_code

then inside docker-compose.yml (version 3.7) or higher Inside the service

    volumes:
      - type: volume
        source: code
        target: /var/www/html

Outside

volumes:
  code:
    external: true
    name: ${PROJECT_VOLUME_CODE}

Answer 4

I managed to use environment variables with this syntax:

volumes:
    - "$HOME/.cache/composer:/tmp/composer_cache"

Answer 5

If you need to change complex stuff between environments, like using completely different volume settings, you should override your configuration with multiple docker-compose files.

Environment variables should be used for simple values only.

Using multiple configuration files lets you define a default docker-compose.yml file with your base configuration, and another docker-compose.override.yml for the changes you need for a particular environment.

Then, when creating the services, docker compose will merge the configuration files.

In your case, your default configuration might look like this:

# docker-compose.yml
version: '3.3'
services:
  target:
    image: "my-registry/my-image:${IMAGE_TAG}"
    volumes:
        - type: volume
          source: vprod
          target: /data
    ports:
     - "80:8080"

volumes:
  vprod:

And your dev override file may look like this:

# docker-compose.override.yml
services:
  target:
    volumes:
        - source: vdev
          target: /data
    ports:
     - "8080:8080"

volumes:
  vdev:

Notice that not all services and not all keys need to be repeated in the override file.

When you run docker-compose up both configurations will be merged with the override file taking precedence.

Docker compose picks up docker-compose.yml and docker-compose.override.yml by default, if you have more files, or files with different names, you need to specify them in order:

docker-compose -f docker-compose.yml -f docker-compose.custon.yml -f docker-compose.dev.yml up -d