How can I use environment variables in docker-compose?

Question

I would like to be able to use environment variables inside docker-compose.yml, with values passed in at the time of docker-compose up. This is the example.

I am doing this today with a basic docker run command, which is wrapped around my own script. Is there a way to achieve it with compose, without any such bash wrappers?

proxy:
  hostname: $hostname
  volumes:
    - /mnt/data/logs/$hostname:/logs
    - /mnt/data/$hostname:/data

For different options see: https://docs.docker.com/compose/environment-variables/ — Masood Khaari, Sep 28 '16 at 09:48
It's solved in the last version of compose, your example will work just as is. check https://docs.docker.com/compose/compose-file/#variable-substitution about variable substitution. — natbusa, Oct 31 '16 at 12:38
Don't forget docker-app (since June 2018): https://stackoverflow.com/a/51007138/6309 — VonC, Jun 24 '18 at 05:20

score 373 · Answer 1 · edited Feb 17 '23 at 01:51

The Docker solution:

Docker-compose 1.5+ has enabled variables substitution: Releases · docker/compose

The latest Docker Compose allows you to access environment variables from your compose file. So you can source your environment variables, then run Compose like so:

set -a
source .my-env
docker-compose up -d

For example, assume we have the following .my-env file:

POSTGRES_VERSION=14

(or pass them via command-line arguments when calling docker-compose, like so: POSTGRES_VERSION=14 docker-compose up -d)

Then you can reference the variables in docker-compose.yml using a ${VARIABLE} syntax, like so:

db:
  image: "postgres:${POSTGRES_VERSION}"

And here is more information from the documentation, taken from Compose file specification

When you run docker-compose up with this configuration, Compose looks for the POSTGRES_VERSION environment variable in the shell and substitutes its value in. For this example, Compose resolves the image to postgres:9.3 before running the configuration.

If an environment variable is not set, Compose substitutes with an empty string. In the example above, if POSTGRES_VERSION is not set, the value for the image option is postgres:.

Both $VARIABLE and ${VARIABLE} syntax are supported. Extended shell-style features, such as ${VARIABLE-default} and ${VARIABLE/foo/bar}, are not supported.

If you need to put a literal dollar sign in a configuration value, use a double dollar sign ($$).

The feature was added in this pull request.

Alternative Docker-based solution: Implicitly sourcing an environment variables file through the `docker-compose` command

If you want to avoid any Bash wrappers, or having to source a environment variables file explicitly (as demonstrated above), then you can pass a --env-file flag to the docker-compose command with the location of your environment variable file: Use an environment file

Then you can reference it within your docker-compose command without having to source it explicitly:

docker-compose --env-file .my-env  up -d

If you don't pass a --env-file flag, the default environment variable file will be .env.

Note the following caveat with this approach:

Values present in the environment at runtime always override those defined inside the .env file. Similarly, values passed via command-line arguments take precedence as well.

So be careful about any environment variables that may override the ones defined in the --env-file!

The Bash solution:

I notice that Docker's automated handling of environment variables can cause confusion. Instead of dealing with environment variables in Docker, let's go back to basics, like Bash! Here is a method using a Bash script and a .env file, with some extra flexibility to demonstrate the utility of environment variables:

POSTGRES_VERSION=14
# Note that the variable below is commented out and will not be used:
# POSTGRES_VERSION=15

# You can even define the compose file in an environment variable like so:
COMPOSE_CONFIG=my-compose-file.yml
# You can define other compose files, and just comment them out
# when not needed:
# COMPOSE_CONFIG=another-compose-file.yml

Then run this Bash script in the same directory, which should deploy everything properly:

#!/bin/bash

docker rm -f `docker ps -aq -f name=myproject_*`
set -a
source .env
cat ${COMPOSE_CONFIG} | envsubst | docker-compose -f - -p "myproject" up -d

Just reference your environment variables in your compose file with the usual Bash syntax (ie ${POSTGRES_VERSION} to insert the POSTGRES_VERSION from the .env file).

While this solution involves Bash, some may prefer it because it has better separation of concerns.

Note the COMPOSE_CONFIG is defined in my .env file and used in my Bash script, but you can easily just replace {$COMPOSE_CONFIG} with the my-compose-file.yml in the Bash script.

Also note that I labeled this deployment by naming all of my containers with the "myproject" prefix. You can use any name you want, but it helps identify your containers so you can easily reference them later. Assuming that your containers are stateless, as they should be, this script will quickly remove and redeploy your containers according to your .env file parameters and your compose YAML file.

Since this answer seems pretty popular, I wrote a blog post that describes my Docker deployment workflow in more depth: Let's Deploy! (Part 1) This might be helpful when you add more complexity to a deployment configuration, like Nginx configurations, Let's Encrypt certificates, and linked containers.

You can simply `grep foo file.text` instead of `cat file.text | grep foo`. In my case I had to `export $(grep "^[^#]" .config | xargs) && cat docker-compose.yml | envsubst`. — Jorge Lavín, Jan 27 '16 at 09:25
"I notice folks have issues with Docker's environment variables support" -- do you have any details or a link to a ticket? — tleyden, Feb 27 '16 at 00:00
Sorry, I didn't log the specific issue I was experiencing, and it was so long ago (~6 months), I don't know whether it's still relevant. But yes, some features in the Docker environment variable support were buggy, and it was reported by multiple users. I believe it is much better now. But when the deployment configuration becomes significantly complex, I would prefer to modularize it by using bash to handle to configuration logic and Docker Compose for the container orchestration. — modulitos, Apr 10 '16 at 22:39
Unless Docker Compose can handle the variety and complexity of configuration possibilities, I think modularizing the config logic separately is more inline with the [Unix philosophy](https://en.wikipedia.org/wiki/Unix_philosophy). And I think bash is a great way to handle this problem through the use of config templates and environment variables. — modulitos, Apr 10 '16 at 22:39
PSA: This **only works with `docker-compose up`** and not with `docker-compose run`. — halbgut, Dec 09 '16 at 09:22
In format 2.1 of the docker-compose file there is also support for default values if the environment variable is not set: `${VARIABLE:-default}`. Link to documentation [here](https://docs.docker.com/compose/compose-file/#variable-substitution). — Matthias Braun, Jan 19 '17 at 17:36
There is this solution https://docs.docker.com/compose/compose-file/#envfile that I use where you add environment variables from a `.env` under `env_file`. Then you can reference the variables in `docker-compose.yml` using `${VARIABLE}` — musale, Mar 29 '17 at 05:39
This answer is out of date. You no longer have to SOURCE the environment script. docker-compose does it for you. https://docs.docker.com/compose/environment-variables/#substitute-environment-variables-in-compose-files — ndemarco, May 21 '21 at 18:19
Perhaps it's worth to note that these environment variables are for a container launch and not for image building (i.e. `up --build` or Dockerfile). For example, for building: https://stackoverflow.com/a/41848041/5113030 (*Use docker-compose env variable in Dockerbuild file*...) — Artfaith, Sep 18 '22 at 11:37
This can get wierd very fast! Check [my example to prevent confusion](https://stackoverflow.com/a/74544228/6474744) — Pedram, Nov 23 '22 at 09:15
and it is called `docker compose` these days. What is super confusing is what one reads in the [docker documentation](https://docs.docker.com/compose/environment-variables/set-environment-variables/) which uses a different (` - `) syntax for `environment:` — theking2, Apr 04 '23 at 14:56

score 371 · Answer 2 · edited Feb 17 '23 at 01:58

371

It seems that docker-compose has native support now for default environment variables in a file.

All you need to do is declare your variables in a file named .env and they will be available in docker-compose.yml.

For example, for a .env file with contents:

MY_SECRET_KEY=SOME_SECRET
IMAGE_NAME=docker_image

You could access your variable inside docker-compose.yml or forward them into the container:

my-service:
  image: ${IMAGE_NAME}
  environment:
    MY_SECRET_KEY: ${MY_SECRET_KEY}

edited Feb 17 '23 at 01:58

Peter Mortensen

30,738
21
105
131

answered May 06 '18 at 15:11

Doody P

4,611
2
20
18

23

this is the best solution! – Ladenkov Vladislav Jan 07 '19 at 22:57
14

This worked for me as well. I don't know why but the file name should be literally `.env`, for example `config.env` did not worked for me. – HBat Mar 09 '19 at 22:21
1

@HBat the "." means a hidden file -- this is usual procedure for local config files – Jeremy Hajek Jun 15 '19 at 16:17
3

The best solution. and We could add /etc/environment props and use those as an environment with using .env. That'll be more secure. – Chinthaka Dinadasa Jun 26 '19 at 09:30
17

@HBat, you can use `env_file` for that, e.g.: `env_file: config.env` – Islam Jun 02 '21 at 23:13
Be aware that `Values set in the shell environment override those set in the .env file.`. That means: default values for environment variables should be in `.env`, if you want to overwrite them look for modulitos answer. – TimFinnegan Jan 01 '22 at 19:18
8

*.env* is for variables that are parsed in to the *docker-compose.yml* interpreter, not for the container. For variables to be set in the container, you will need to specify a *.env* file in your *docker-compose.yml* using `env_file: .env.vars` or whatever your file name is that contains the variables. It's a confusing antipattern to use `env_file: .env` which works but basically uses the variables both for the interpreter and the container. – Peter Kionga-Kamau Oct 05 '22 at 22:56
@PeterKionga-Kamau But is this really always an anti-pattern? I'd rather have people who want to deploy my application only set up *one* .env file (e.g. from a template) than instructing them to but certain variables in certain files. It's *my* job to choose the variable names in a way there are no conflicts then. Basically, if I know what I'm doing, setting `env_file: .env` can make the whole setup much more "streamlined". – bkis Apr 19 '23 at 08:36
1

@bkis It's an anti-pattern because despite appearing to be easier and/or adequate, proper use requires knowledge that is not obvious or well documented, and can lead to confusion (few people are aware that there are actually separate files for the yaml parse and the build part until they run into issues). There are also security concerns; it's generally poor practice to allow variables that are no longer needed to persist through to the final build image/containers. – Peter Kionga-Kamau Apr 20 '23 at 16:40

score 120 · Accepted Answer · edited Mar 26 '23 at 12:30

120

Create a template.yml, which is your docker-compose.yml with environment variable.
Suppose your environment variables are in a file 'env.sh'
Put the below piece of code in a sh file and run it.

source env.sh
rm -f docker-compose.yml
envsubst < "template.yml" > "docker-compose.yml"

A new file docker-compose.yml will be generated with the correct values of environment variables.

Sample template.yml file:

oracledb:
        image: ${ORACLE_DB_IMAGE}
        privileged: true
        cpuset: "0"
        ports:
                - "${ORACLE_DB_PORT}:${ORACLE_DB_PORT}"
        command: /bin/sh -c "chmod 777 /tmp/start; /tmp/start"
        container_name: ${ORACLE_DB_CONTAINER_NAME}

Sample env.sh file:

#!/bin/bash 
export ORACLE_DB_IMAGE=<image-name> 
export ORACLE_DB_PORT=<port to be exposed> 
export ORACLE_DB_CONTAINER_NAME=ORACLE_DB_SERVER

edited Mar 26 '23 at 12:30

Andreas Violaris

2,465
5
13
26

answered Dec 07 '15 at 09:12

Saurabh Kumar

1,506
1
11
7

13

still no better solution at the moment? – lvthillo Jan 12 '16 at 10:43
26

why would you recursively delete a file? (rm -rf docker-compose.yml) – moritzschaefer Aug 26 '16 at 17:19
22

-1 this solution only complicates things and should be updated according to docker-compose new abilities https://docs.docker.com/compose/environment-variables/#substitute-environment-variables-in-compose-files – Efrat Levitan Aug 01 '19 at 07:20
3

I would recommend using the above command inside the parenthesis so that even the variable will not be set globally. Ex. `(source env.sh; rm -rf docker-compose.yml; envsubst < "template.yml" > "docker-compose.yml";)` – Ishtiyaq Husain May 16 '20 at 22:22
5

This answer is out of date. – ndemarco May 21 '21 at 18:17

score 72 · Answer 4 · answered Oct 20 '17 at 16:34

72

The best way is to specify environment variables outside the docker-compose.yml file. You can use env_file setting, and define your environment file within the same line. Then doing a docker-compose up again should recreate the containers with the new environment variables.

Here is how my docker-compose.yml looks like:

services:
  web:
    env_file: variables.env

Note: docker-compose expects each line in an env file to be in VAR=VAL format. Avoid using export inside the .env file. Also, the .env file should be placed in the folder where the docker-compose command is executed.

answered Oct 20 '17 at 16:34

Jibu James

1,581
14
11

8

Best way indeed – Dany Mar 06 '20 at 16:09
9

NO. It will not automatically make the environment variables available inside the docker container. You still need to explicitly mention these under environment section. – kta Jun 04 '20 at 05:16
It worked smoothly in my case, maybe it has to do with the docker compose file version. I was using version 3 – Zisis F Oct 08 '21 at 12:18
Worked well for me. Docker compose 2.7.0 – Rob Sanders Jul 20 '22 at 19:31

Pedram · Answer 5 · 2023-07-23T08:14:58.927

Don't confuse the `.env` file and the `env_file` option!

They serve totally different purposes!

The .env file feeds those environment variables only to your docker compose file, which in turn, can be passed to the containers as well.

But the env_file option only passes those variables to the containers and NOT the docker compose file ‍

IMPORTANT NOTE ON SECRETS

The example is using the environment variables to pass secret credentials to the docker compose file. There is a better encrypted and safe way to do this explained in detail on official docs: How to use secrets in Docker Compose

Example

OK, let's say we have this simple compose file:

services:
  foo:
    image: ubuntu
    hostname: suchHostname # <-------------- hard coded 'suchHostname'
    volumes:
      - /mnt/data/logs/muchLogs:/logs # <--- hard coded 'muchLogs'
      - /mnt/data/soBig:/data # <----------- hard coded 'soBig'

We don't want to hard code these anymore! So, we can put them in the current terminal's environment variables and check if docker-compose understands them:

$ export the_hostname="suchHostName"
$ export dir_logs="muchLogs"
$ export dir_data="soBig"

and change the docker-compose.yml file to:

services:
  foo:
    image: ubuntu
    hostname: $the_hostname # <-------------- use $the_hostname
    volumes:
      - /mnt/data/logs/$dir_logs:/logs # <--- use $dir_logs
      - /mnt/data/$dir_data:/data # <-------- usr $dir_data

Now let's check out if it worked with executing $ docker-compose config and inspecting the output:

name: tmp
services:
  foo:
    hostname: suchHostName # <------------- $the_hostname
    image: ubuntu
    networks:
      default: null
    volumes:
    - type: bind
      source: /mnt/data/logs/muchLogs # <-- $dir_logs
      target: /logs
      bind:
        create_host_path: true
    - type: bind
      source: /mnt/data/soBig # <---------- $dir_data
      target: /data
      bind:
        create_host_path: true
networks:
  default:
    name: tmp_default

OK it works! But let's use the .env file instead. Since docker-compose understands the .env file, let's just create one and set it up:

# .env file (in the same directory as 'docker-compose.yml')
the_hostname="suchHostName"
dir_logs="muchLogs"
dir_data="soBig"

OK, you can test it with a NEW terminal (so that the older environment variables we set with export don't interfere and make sure everything works in a clean terminal) Just follow step 4 again and see that it works!

So far so good However, when you stumble upon the env_file option, it gets confusing Let's say that you want to pass a password to the docker compose file (NOT the container).

In the wrong approach, you might put a password in .secrets file:

# .secrets
somepassword="0P3N$3$@M!"

and then update the docker-compose file as follows:

services:
  foo:
    image: ubuntu
    hostname: $the_hostname
    volumes:
      - /mnt/data/logs/$dir_logs:/logs
      - /mnt/data/$dir_data:/data

    #  BAD: 
    env_file:
      - .env
      - .secrets
    entrypoint: echo "Hush! This is a secret '$somepassword'"

Now checking it just like step 4 again would result in:

WARN[0000] The "somepassword" variable is not set. Defaulting to a blank string. 
name: tmp                       #       ^
services:                       #       |
  foo:                          #       |
    entrypoint:                 #       |
    - echo                      #       |
    - Hush! This is a secret '' # <---- ‍ Oh no!
    environment:
      dir_data: soBig
      dir_logs: muchLogs
      somepassword: 0P3N$$3$$@M! # <---  Huh?!
      the_hostname: suchHostName
    hostname: suchHostName
    image: ubuntu
    networks:
      default: null
    volumes:
    - type: bind
      source: /mnt/data/logs/muchLogs
      target: /logs
      bind:
        create_host_path: true
    - type: bind
      source: /mnt/data/soBig
      target: /data
      bind:
        create_host_path: true
networks:
  default:
    name: tmp_default

So as you can see, the $somepassord variable is only passed to the container, and NOT the docker compose file.

Wrapping up

You can pass environment variables to docker-compose files in two ways:

By exporting the variable to the terminal before running docker compose.
By putting the variables inside .env file.

The env_file option only passes those extra variables to the containers and not the compose file

Useful links

Using different environment files for production and development with the --env-file command, more on Ways to set environment variables in Compose
Since this is important, again, to SAFELY pass the secrets to containers, read Use secrets in Docker Compose
Use .config files, so that there is no need to rebuild Docker images. More on config top-level element
Something weird happened when you set up the same variable in different environment files? This might help: Environment Variables Precedence

Title is very explanatory and should be basic knowledge for every developer working with Docker. It wasn't for me until now — Jos, Dec 06 '22 at 12:02
Yes I learned it the hard way as well Actually, I researched a bit more after reading answers in this page, and added what was missing — Pedram, Dec 08 '22 at 17:42
Use **`docker-compose config` instead of** `docker-compose convert` for more recent versions — Maicon Mauricio, Feb 28 '23 at 16:30
Ok but I want to keep ALL my env files in a separate dedicated directory, how can I tell compose to read variables from a custom location? It seems that it has been hardcoded to read variables only from the same directory? — ruslaniv, Jul 21 '23 at 09:09
@ruslaniv Hey there. - You can specify the path with `--env-file` (more on https://docs.docker.com/compose/environment-variables/set-environment-variables/#substitute-with---env-file). - If you are trying to keep different secret variables separate, please use the `secrets` approach explained here: https://docs.docker.com/compose/use-secrets/ - For different configs (so that you wouldn't need to rebuild the docker image), use: https://docs.docker.com/compose/compose-file/08-configs/ — Pedram, Jul 23 '23 at 07:58

score 47 · Answer 6 · answered Jan 03 '19 at 13:18

47

The following is applicable for docker-compose 3.x Set environment variables inside the container

method - 1 Straight method

web:
  environment:
    - DEBUG=1
      POSTGRES_PASSWORD: 'postgres'
      POSTGRES_USER: 'postgres'

method - 2 The “.env” file

Create a .env file in the same location as the docker-compose.yml

$ cat .env
TAG=v1.5
POSTGRES_PASSWORD: 'postgres'

and your compose file will be like

$ cat docker-compose.yml
version: '3'
services:
  web:
    image: "webapp:${TAG}"
    postgres_password: "${POSTGRES_PASSWORD}"

source

answered Jan 03 '19 at 13:18

Gajendra D Ambi

3,832
26
30

4

I'd like to see a complete example of method 1. I couldn't make this work, so ended up using .env file (which worked fine). – BobHy May 25 '19 at 01:02
2

What is the difference between `- DEBUG=1` and `DEBUG: 1` methods of variable assignments ? – Gagik Aug 18 '22 at 09:25

score 39 · Answer 7 · answered Feb 26 '18 at 13:29

39

When using environment variables for volumes you need:

create .env file in the same folder which contains docker-compose.yaml file
declare variable in the .env file:
```
HOSTNAME=your_hostname
```

Change $hostname to ${HOSTNAME} at docker-compose.yaml file

proxy:
  hostname: ${HOSTNAME}
  volumes:
    - /mnt/data/logs/${HOSTNAME}:/logs
    - /mnt/data/${HOSTNAME}:/data

Of course you can do that dynamically on each build like:

echo "HOSTNAME=your_hostname" > .env && sudo docker-compose up

answered Feb 26 '18 at 13:29

Nerijus G

918
13
28

17

Note, according to the docs: `The .env file feature only works when you use the docker-compose up command and does not work with docker stack deploy.` – James Gentes Apr 05 '18 at 00:33
1

This needs to be in the CONTEXT directory NOT the `docker-compose.yaml` directory – Joe Sep 24 '20 at 03:07
2

@Joe, this comment saved my life so to speak. Couldn't figure out why .env file was being ignored. – Yuri Pozniak Oct 17 '20 at 14:58

score 11 · Answer 8 · answered May 11 '20 at 12:10

11

Since 1.25.4, docker-compose supports the option --env-file that enables you to specify a file containing variables.

Yours should look like this:

hostname=my-host-name

And the command:

docker-compose --env-file /path/to/my-env-file config

answered May 11 '20 at 12:10

papillon

1,807
2
19
39

score 8 · Answer 9 · edited Feb 17 '23 at 02:04

To add an environment variable, you may define an env_file (let's call it var.env) as:

ENV_A=A
ENV_B=B

And add it to the docker compose manifest service. Moreover, you can define environment variables directly with environment.

For instance, in docker-compose.yaml:

version: '3.8'
services:
  myservice:
    build:
      context: .
      dockerfile: ./docker/Dockerfile.myservice
    image: myself/myservice
    env_file:
     - ./var.env
    environment:
     - VAR_C=C
     - VAR_D=D
    volumes:
     - $HOME/myfolder:/myfolder
    ports:
     - "5000:5000"

Please check here for more/updated information: Manuals → Docker →Compose →Environment variables → Overview

score 7 · Answer 10 · edited Feb 17 '23 at 02:01

Use:

env SOME_VAR="I am some var" OTHER_VAR="I am other var" docker stack deploy -c docker-compose.yml

Use version 3.6:

version: "3.6"
services:
  one:
    image: "nginx:alpine"
    environment:
      foo: "bar"
      SOME_VAR:
      baz: "${OTHER_VAR}"
    labels:
      some-label: "$SOME_VAR"
  two:
    image: "nginx:alpine"
    environment:
      hello: "world"
      world: "${SOME_VAR}"
    labels:
      some-label: "$OTHER_VAR"

I got it from Feature request: Docker stack deploy pass environment variables via cli options #939.

Thomas Decaux · Answer 11 · 2015-04-23T17:38:32.017

6

You cannot ... yet. But this is an alternative, think like a docker-composer.yml generator:

https://gist.github.com/Vad1mo/9ab63f28239515d4dafd

Basically a shell script that will replace your variables. Also you can use Grunt task to build your docker compose file at the end of your CI process.

edited Apr 23 '15 at 17:38

answered Apr 23 '15 at 17:25

Thomas Decaux

21,738
2
113
124

score 5 · Answer 12 · answered May 21 '15 at 08:48

I have a simple bash script I created for this it just means running it on your file before use: https://github.com/antonosmond/subber

Basically just create your compose file using double curly braces to denote environment variables e.g:

app:
    build: "{{APP_PATH}}"
ports:
    - "{{APP_PORT_MAP}}"

Anything in double curly braces will be replaced with the environment variable of the same name so if I had the following environment variables set:

APP_PATH=~/my_app/build
APP_PORT_MAP=5000:5000

on running subber docker-compose.yml the resulting file would look like:

app:
    build: "~/my_app/build"
ports:
    - "5000:5000"

score 4 · Answer 13 · edited Feb 17 '23 at 01:55

Add an environment variable to the .env file

Such as

VERSION=1.0.0

Then save it to `deploy.sh`

INPUTFILE=docker-compose.yml
RESULT_NAME=docker-compose.product.yml
NAME=test

prepare() {
        local inFile=$(pwd)/$INPUTFILE
        local outFile=$(pwd)/$RESULT_NAME
        cp $inFile $outFile
        while read -r line; do
            OLD_IFS="$IFS"
            IFS="="
            pair=($line)
            IFS="$OLD_IFS"
               sed -i -e "s/\${${pair[0]}}/${pair[1]}/g" $outFile
            done <.env
     }

deploy() {
        docker stack deploy -c $outFile $NAME
}


prepare
deploy

score 4 · Answer 14 · answered May 18 '21 at 14:42

To focus solely on the issue of default and mandatory values for environment variables, and as an update to @modulito's answer:

Using default values and enforcing mandatory values within the docker-compose.yml file is now supported (from the docs):

Both $VARIABLE and ${VARIABLE} syntax are supported. Additionally when using the 2.1 file format, it is possible to provide inline default values using typical shell syntax:

${VARIABLE:-default} evaluates to default if VARIABLE is unset or empty in the environment. ${VARIABLE-default} evaluates to default only if VARIABLE is unset in the environment.

Similarly, the following syntax allows you to specify mandatory variables:

${VARIABLE:?err} exits with an error message containing err if VARIABLE is unset or empty in the environment. ${VARIABLE?err} exits with an error message containing err if VARIABLE is unset in the environment.

Other extended shell-style features, such as ${VARIABLE/foo/bar}, are not supported.

score 4 · Answer 15 · answered Jun 16 '22 at 10:12

This was written for Docker v20, using the docker compose v2 commands.

I was having a similar roadblock and found that the --env-file parameter ONLY works for docker compose config command. On top of that using the docker compose env_file variable, still forced me to repeat values for the variables, when wanting to reuse them in other places than the Dockerfile such as environment for docker-compose.yml. I just wanted one source of truth, my .env, with the ability to swap them per deployment stage. So here is how I got it to work, basically use docker compose config to generate a base docker-compose.yml file that will pass ARG into Dockerfile's.

.local.env This would be your .env, I have mine split for different deployments.

DEVELOPMENT=1
PLATFORM=arm64

docker-compose.config.yml - This is my core docker compose file.

services:
  server:
    build:
      context: .
      dockerfile: docker/apache2/Dockerfile
      args:
        - PLATFORM=${PLATFORM}
        - DEVELOPMENT=${DEVELOPMENT}
    environment:
      - PLATFORM=${PLATFORM}
      - DEVELOPMENT=${DEVELOPMENT}

Now sadly I do need to pass in the variables twice, once for the Dockerfile, the other for environment. However, they are still coming from the single source .local.env so at least I do not need to repeat values.

I then use docker compose config to generate a semi-final docker-compose.yml. This lets me pass in my companion override docker-compose.local.yml for where the final deployment is happening.

docker compose --env-file=.local.env -f docker-compose.config.yml config > docker-compose.yml

This will now let my Dockerfile access the .env variables.

FROM php:5.6-apache

# Make sure to declare after FROM
ARG PLATFORM
ARG DEVELOPMENT

# Access args in strings with $PLATFORM, and can wrap i.e ${PLATFORM}
RUN echo "SetEnv PLATFORM $PLATFORM" > /etc/apache2/conf-enabled/environment.conf
RUN echo "SetEnv DEVELOPMENT $DEVELOPMENT" > /etc/apache2/conf-enabled/environment.conf

This then passes the .env variables from the docker-compose.yml into Dockerfile which then passes it into my Apache HTTP server, which passes it to my final destination, the PHP code.

My next step to then to pass in my docker compose overrides from my deployment stage.

docker-compose.local.yml - This is my docker-compose override.

services:
  server:
    volumes:
      - ./localhost+2.pem:/etc/ssl/certs/localhost+2.pem
      - ./localhost+2-key.pem:/etc/ssl/private/localhost+2-key.pem

Lastly, run the docker compose command.

docker compose -f docker-compose.yml -f docker-compose.local.yml up --build

Please note if you change anything in you .env file you will need to re-run the docker compose config and add --build for docker compose up. Since builds are cached it has little impact.

So for my final command I normally run:

docker compose --env-file=.local.env -f docker-compose.config.yml config > docker-compose.yml; docker compose --env-file=.local.env -f docker-compose.yml -f docker-compose.local.yml up --build

score 3 · Answer 16 · answered Jul 20 '15 at 12:09

3

As far as I know, this is a work-in-progress. They want to do it, but it's not released yet. See 1377 (the "new" 495 that was mentioned by @Andy).

I ended up implementing the "generate .yml as part of CI" approach as proposed by @Thomas.

answered Jul 20 '15 at 12:09

anroots

1,959
1
14
21

score 2 · Answer 17 · edited Feb 17 '23 at 02:08

2

It's simple like this:

Using command line as mentioned in the documentation:

docker-compose --env-file ./config/.env.dev config

Or using a .env file, I think this is the easiest way:

 web:
  env_file:
    - web-variables.env

Documentation with a sample

edited Feb 17 '23 at 02:08

Peter Mortensen

30,738
21
105
131

answered May 27 '22 at 13:53

Fernando Nogueira

1,302
1
14
22

sorabzone · Answer 18 · 2018-10-06T18:33:38.693

Use .env file to define dynamic values in docker-compse.yml. Be it port or any other value.

Sample docker-compose:

testcore.web:
       image: xxxxxxxxxxxxxxx.dkr.ecr.ap-northeast-2.amazonaws.com/testcore:latest
       volumes: 
            - c:/logs:c:/logs
       ports:
            - ${TEST_CORE_PORT}:80
       environment:
            - CONSUL_URL=http://${CONSUL_IP}:8500 
            - HOST=${HOST_ADDRESS}:${TEST_CORE_PORT}

Inside .env file you can define the value of these variables:

CONSUL_IP=172.31.28.151
HOST_ADDRESS=172.31.16.221
TEST_CORE_PORT=10002

score 1 · Answer 19 · answered May 13 '21 at 14:35

I ended up using "sed" in my deploy.sh script to accomplish this, though my requirements were slightly different since docker-compose is being called by Terrafom: Passing Variables to Docker Compose via a Terraform script for an Azure App Service

eval "sed -i 's/MY_VERSION/$VERSION/' ../docker-compose.yaml"

cat ../docker-compose.yaml

terraform init 
terraform apply -auto-approve \
    -var "app_version=$VERSION" \
    -var "client_id=$ARM_CLIENT_ID" \
    -var "client_secret=$ARM_CLIENT_SECRET" \
    -var "tenant_id=$ARM_TENANT_ID" \
    -var "subscription_id=$ARM_SUBSCRIPTION_ID"

eval "sed -i 's/$VERSION/MY_VERSION/' ../docker-compose.yaml"

score 0 · Answer 20 · answered Jun 07 '23 at 04:27

Solution I was seeking for an use case where I can utilize environment variable in command: of docker-compose.yml.

To access environment variable use $$STREAM_VERSION

version: '3.0'
services:
  coverity:
    environment:
      - STREAM_VERSION=CCD
    volumes:
      - $GOPATH/src/gitlab.protectv.local/ddx/:/go/Coverity/
    tty: true 
    build:
      context: .
    image: coverity
    restart: on-failure
    ports:
      - "7777:7777"
    container_name: coverity-container
    # TODO: Dynamic value for build version in cov-commit-defects
    command: sh -c "echo $$STREAM_VERSION && cov-build --dir $GOPATH/cvbuild go install && cov-analyze --all --dir $GOPATH/cvbuild --strip-path $GOPATH && cd /go/Coverity/ddf.git && cov-commit-defects --url https://coverity-sj.thalesesec.com:8443 --stream $${STREAM_VERSION} --dir $GOPATH/cvbuild --auth-key-file /usr/local/cov-analysis-linux64-2022.3.0/bin/auth.file"
    
volumes:
  coverity-data:
    #driver: local

Hope it helps to future readers.

How can I use environment variables in docker-compose?

20 Answers20

The Docker solution:

Alternative Docker-based solution: Implicitly sourcing an environment variables file through the `docker-compose` command

The Bash solution:

Don't confuse the `.env` file and the `env_file` option!

IMPORTANT NOTE ON SECRETS

Example

Wrapping up

Useful links

Add an environment variable to the .env file

Then save it to `deploy.sh`

Linked

How can I use environment variables in docker-compose?

20 Answers20

The Docker solution:

Alternative Docker-based solution: Implicitly sourcing an environment variables file through the docker-compose command

The Bash solution:

Don't confuse the .env file and the env_file option!

IMPORTANT NOTE ON SECRETS

Example

Wrapping up

Useful links

Add an environment variable to the .env file

Then save it to deploy.sh

Linked

Alternative Docker-based solution: Implicitly sourcing an environment variables file through the `docker-compose` command

Don't confuse the `.env` file and the `env_file` option!

Then save it to `deploy.sh`