Docker-compose env file not working

Question

I'm writing as docker-compose file to up MySQL instance and want to use few variable from env file: here are the files actually look like:

docker-compose.yml

version: '3.3'
services:
  db:
    image: mysql
    restart: always
    env_file:
      - ./imran.env
    environment:
      MYSQL_ROOT_PASSWORD: ${PASS}
    ports:
      - ${PORT1}: ${PORT2}

imran.env

PASS=imran123
PORT1=3306
PORT2=3306

Instead of working correct i'm getting following errors:

WARNING: The PASS variable is not set. Defaulting to a blank string.
WARNING: The PORT2 variable is not set. Defaulting to a blank string.
ERROR: The Compose file './docker-compose.yml' is invalid because:
services.db.ports contains unsupported option: '${PORT1}

Please Help

score 67 · Answer 1 · answered Jan 29 '18 at 11:17

67

You have some issues in your docker-compose.yaml file:

A:

A space symbol between ports values. It should be without a space:

ports:
  - ${PORT1}:${PORT2}

B:

You need to use .env file in folder where docker-compose.yaml is in order to declaring default environment variables for both docker-compose.yaml file and docker container. env_file section is used to put values into container only.

So, you should do the following:

1.

Re-name file with ENV variables to .env:

mv imran.env .env

2.

Use the following docker-compose.yaml after:

version: '3.3'
services:
  db:
    image: mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: ${PASS}
    ports:
      - ${PORT1}:${PORT2}

answered Jan 29 '18 at 11:17

nickgryg

25,567
5
77
79

57

@imranahmedani maybe the courtesy of accepting the answer? – Tim Oct 12 '18 at 08:26
14

What about the real answer about env_file option, where I can specify the external file path, it is not working? – Deadpool Jul 20 '20 at 17:48
21

Downvoting. Not the right answer. The question is about how to specify an environment file explicitly. – Mudassir Shahzad Nov 04 '20 at 07:28
3

> "env_file section is used to put values into container only" / That's really a fail. They should have created an option to specify custom name (and dir) for `.env` file... – greatvovan Jan 21 '21 at 23:30

score 47 · Answer 2 · answered May 24 '22 at 19:50

47

The .env file in the project root, and the env_file: field in the Compose file are two different concepts.

The .env is for settings a default environment for Compose. Values set in this file can be used within the Compose file.

The env_file: field is for setting the default environment for a container. Values set in this can be used in the container, but not in the Compose file.

See https://docs.docker.com/compose/env-file/ for more information.

answered May 24 '22 at 19:50

Codertjay

588
8
13

2

I guess this should be the accepted answer. To add to that then I assume `docker-compose --env-file` option is actually for setting the default environment for Compose – BozanicJosip Jul 25 '22 at 09:48
1

this answer right here folks – danludwig Mar 10 '23 at 20:02
5

My goodness, thank you! The official Docker docs lists both options when talking about "Environment variables precedence" as if they were fulfilling the same purpose, just their precedence was different. But they are not interchangeable in the slightest! https://docs.docker.com/compose/environment-variables/envvars-precedence/ – DanielM Apr 18 '23 at 12:37

score 35 · Answer 3 · answered Nov 10 '20 at 23:20

It seems that env_file is ignored when you add environment after it. In my experience, docker-compose seems to have numerous "silly" bugs like this.

Your docker-compose.yaml has a few errors which are irrelevant to env files. I corrected them a bit.

$ tre
.
├── broken
│   ├── imran.env
│   └── docker-compose.yaml
└── works
    ├── docker-compose.yaml
    └── imran.env

$ bat broken/*
───────┬──────────────────────────────────────────────
       │ File: broken/docker-compose.yaml
───────┼──────────────────────────────────────────────
   1   │ version: '3.3'
   2   │ services:
   3   │   db:
   4   │     image: mysql
   5   │     restart: always
   6   │     env_file:
   7   │       - ./imran.env
   8   │     environment:
   9   │       MYSQL_ROOT_PASSWORD: ${PASS}
───────┴──────────────────────────────────────────────
───────┬──────────────────────────────────────────────
       │ File: broken/imran.env
───────┼──────────────────────────────────────────────
   1   │ PASS=imran123
   2   │ MYSQL_ROOT_PASSWORD=changeme
   3   │ PORT1=3306
   4   │ PORT2=3306
───────┴──────────────────────────────────────────────

$ diff broken works
diff broken/docker-compose.yaml works/docker-compose.yaml
8,9d7
<     environment:
<       MYSQL_ROOT_PASSWORD: ${PASS}

With environment, it fails:

$ cd broken && docker-compose down && docker-compose up -d
WARNING: The PASS variable is not set. Defaulting to a blank string.
Stopping broken_db_1 ... done
Removing broken_db_1 ... done
Removing network broken_default
WARNING: The PASS variable is not set. Defaulting to a blank string.
Creating network "broken_default" with the default driver
Creating broken_db_1 ... done

But without environment, it works:

$ cd works && docker-compose down && docker-compose up -d
Removing works_db_1 ... done
Removing network works_default
Creating network "works_default" with the default driver
Creating works_db_1 ... done

$ docker exec -i -t works_db_1 env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=ac76b84ca072
TERM=xterm
PASS=imran123
MYSQL_ROOT_PASSWORD=changeme
PORT1=3306
PORT2=3306
GOSU_VERSION=1.12
MYSQL_MAJOR=8.0
MYSQL_VERSION=8.0.21-1debian10
HOME=/root

Notes:

Docker-compose apparently doesn't like ${PORT1} in ports:, you have to give the actual number. (in this case I just deleted it because ports are irrelevant for env files)
The MySQL image crashes if you don't provide a root pass. I added one so the container could run long enough for me to dump its env and confirm that the variables are being set.
I am using Docker version 19.03.5 on macOS Catalina 10.15.4.

It seems from https://docs.docker.com/compose/environment-variables/#the-env-file. When you set the same environment variable in multiple files, here’s the priority used by Compose to choose which value to use: Compose file, Shell environment variables, Environment file, Dockerfile, Variable is not defined — zhfkt, Jan 29 '21 at 02:58
@zhfkt is right: is not a bug, configuration' `environment` key does the override of `.env` file, it's stated int the official doc. — funder7, Feb 02 '22 at 21:07
@funder7 I don't think so. The docs state what takes precedence "when you set the same environment variable" - which is not the case here. — Eike, Feb 15 '22 at 15:18
@Eike it states "When you run the container, the environment variable defined in the Compose file takes precedence." And also "Values in the shell take precedence over those specified in the .env file." It's explained in [this paragraph](https://docs.docker.com/compose/environment-variables/#set-environment-variables-with-docker-compose-run) — funder7, Feb 15 '22 at 23:01
@funder7 But which is the variable in the Compose file that would take precedence over the respective variable in the env file? All variables seem to be different? — Eike, Feb 18 '22 at 13:26
I think something different is happening. The env_file line is feeding the docker container (docs: "You can pass multiple environment variables from an external file through to a service’s containers[sic]"). But the PASS variable would be needed in docker-compose already, as it's mentioned there. Interestingly, and in opposite to what the docs ("just like with docker run --env-file=FILE ") are saying, "docker-compose --env-file=imran.env up" works for me. — Eike, Feb 18 '22 at 13:56
@Eike the `.env` file is applied to the whole docker project, while in the `docker-compose.yml` you define services (aka containers), each one having it's `environment` property, where you can define which environment variables to set up. Another option is to set the `env-file` property in the container definition, and every variable in the file, will be injected in the container. I hope that now it's a little more clear! — funder7, Feb 21 '22 at 20:48

score 7 · Answer 4 · answered Jun 03 '21 at 05:50

7

I recently had to specify that I wanted my container to use .env for its environment file like:

version: '2.4'

services:

  myapp:
    build:
      context: .
    container_name: myapp01xj1
    env_file:
      - .env

answered Jun 03 '21 at 05:50

Charney Kaye

3,667
6
41
54

score 5 · Answer 5 · answered Dec 13 '21 at 10:57

5

In my case, I was running docker-compose up from a sub directory inside the project(by mistake). So, make sure that the docker-compose.yml file and .env in the same directory, and that you are running docker-compose up in the same directory where both of the files exists.

answered Dec 13 '21 at 10:57

Jehad Nasser

3,227
5
27
39

does it even complain that it can't find the file or there's just some message posted but nothing breaks to disrupt stuff in some obvious way? – jxramos Jun 15 '22 at 18:03
@jxramos As far as I remember, it was same error message as the question here is showing, but it's easy to reproduce and make sure. – Jehad Nasser Jun 19 '22 at 22:19

score 4 · Answer 6 · answered Apr 30 '22 at 07:24

@nickgryg answer is good but here I will explain what I had faced when working with env. It will help others to understand.

I was also not able to access .my-project.env some variables inside docker-compose file.

.my-project.env

ROCKETTYPE=SIMPLE
RANGE=15

docker-compose.yml

   env_file:
      - .my-project.env
    environment:
      - SPACEX:${ROCKETTYPE}
   # It was importing RANGE env not SPACEX inside my container.

When I ran docker-compose up -d, only RANGE env accessible in my container.

Then I came up with two solutions :

1st : Instead of using .my-project.env file only use .env file name in docker-compose.

2nd : I did not change my env file name. Only passing .my-project.env in docker-compose command.

docker-compose --env-file .my-project.env  up -d

Docker-Compose version : docker-compose version 1.25.0

Read official docker documentations

score 2 · Answer 7 · answered Jul 08 '22 at 05:48

2

Use docker compose up Does not work on legacy docker-compose up

answered Jul 08 '22 at 05:48

Руслан Тютин

115
1
7

score 2 · Answer 8 · answered Jul 13 '22 at 14:57

According to this docker manual, if environment variables are used in docker-compose.yml file like:

web:
  image: "webapp:${TAG}"

or

environment:
  MYSQL_ROOT_PASSWORD: ${PASS}

Docker looks for those variables in a default .env file; So changing the default env file like docker-compose up --env-file imran.env would work.

tntnkn · Answer 9 · 2023-04-01T09:48:21.423

It seems like the question have been there for a while, but for a curios posterity (like myself) the correct answer (as yours truly considers the one to be, at least) would be as follows:

As other good people mentioned in this discussion, env_file property of docker-compose.yml is setting environment variables for containers, not for docker-compose file, and is similar to docker run --env-file;
By default, environment variables in docker compose file is set by .env file in the same folder;
To change this default file, start your services with docker compose --env-file specifying an alternative .env file.

The answer comes directly from here.

Also, please note that, as mentioned here, passing an alternative dotenv file overwrites the default .env.

score 1 · Answer 10 · answered Jun 12 '22 at 15:08

i had a similar problem but a bit different.

the mysql container failed to boot up because the .env beside the docker-compose.yml was not being picked up by docker-compose.

this probably happened because i had another file beside them called .env.example which confused docker-compose.

i resolved this by moving the .env.example file to another folder, but i also tried explicitly specifying the .env file in the docker-compose.yml which also worked.

i.e:

    env_file:
      - .env

i did not have a need for an environment key in my docker-compose.yml, which makes my answer less relevant to this question but i think its close enough to help others that find this question.

score 1 · Answer 11 · answered Dec 01 '22 at 15:27

In my case it was enough to add the parameter...

    env_file:
      - ./.env-MY

... in docker-compose.yml for the desired service.

PLUS:

The name of the .env file (.env-MY, my case) is of no importance, it can be any name.

The --env-file ./.env-MY parameter will not be needed for the docker-compose command.

It is NOT necessary to declare environment variables inside docker-compose.yml, as per this example...

    environment:
      - ENV_VAR_A:${ENV_VAR_A}
      - ENV_VAR_B:${ENV_VAR_B}
      - ENV_VAR_C:${ENV_VAR_C}
      - ENV_VAR_D:${ENV_VAR_D}
      - ENV_VAR_E:${ENV_VAR_E}
      - ENV_VAR_F:${ENV_VAR_F}
      - ENV_VAR_G:${ENV_VAR_G}
      - ENV_VAR_H:${ENV_VAR_H}

This is the example content of my .env-MY...

#!/bin/bash

ENV_VAR_A="env_var_a_val"
ENV_VAR_B="env_var_b_val"
ENV_VAR_C="env_var_c_val"
ENV_VAR_D="env_var_d_val"
ENV_VAR_E="env_var_e_val"
ENV_VAR_F="env_var_f_val"
ENV_VAR_G="env_var_g_val"
ENV_VAR_H="env_var_h_val"

To test whether the environment variables were loaded run the command...

docker-compose exec <SERVICE> env

... in the desired service.

[Ref(s).: https://stackoverflow.com/a/34051804/3223785 ]

score 0 · Answer 12 · answered Nov 29 '22 at 12:29

Had a similar issue. I had my environment variables in a .env and was using the following in the docker -compose.yml file.

environment:
  - DJANGO_SECRET_KEY:${DJANGO_SECRET_KEY}
  - ALLOWED_HOSTS:${ALLOWED_HOSTS}

Changed it to

environment:
  - DJANGO_SECRET_KEY=${DJANGO_SECRET_KEY}
  - ALLOWED_HOSTS=${ALLOWED_HOSTS}

And it worked. Notice the symbol : replaced with = Using version 3.8

Docker-compose env file not working

12 Answers12

Linked