How to set environment variables in docker-compose

Question

I have a docker-compose file in which I use env_file to read and set a bunch of env variables at run time. These env variables are required for a command that I need to run at run time using command. However it looks like the command section is executed before the env variables are set at run time and this cause an error. How can I ensure that setting the env variables occur before executing the command section in a docker-compose?

Here is my docker-compose file

services:
  mlx-python-hdfs:
    image: image_name
    container_name: cname
    env_file: ./variables.txt
    command:
         - microservice $VAR1 $VAR2

$VAR1 and $VAR2 are read from variables.txt file but when I start the container it complains on "microservice $VAR1 $VAR2" line and show the $VAR1 and VAR2 as empty.

Could you share your docker-compose / env file? And what makes you think the env is set after your command IMHO such a feature would make no sense, so that would be a bug most likely. — Martin van Wingerden, Sep 03 '19 at 14:27
I edited my original post wil my dockr-compose file. Please have a look. — HHH, Sep 03 '19 at 14:35

score 3 · Accepted Answer · answered Sep 03 '19 at 15:00

3

rename your file to .env (without name) so mv variables.txt .env

edit your compose :

services:
  mlx-python-hdfs:
    image: image_name
    container_name: cname
    command:
         - microservice $VAR1 $VAR2

then run it normally see this

answered Sep 03 '19 at 15:00

LinPy

16,987
4
43
57

score 1 · Answer 2 · answered Sep 03 '19 at 17:20

The Docker Compose command: directive has two forms. If you specify it as a list, it is read as a list of explicit individual arguments; no shell is invoked over it, and there is no argument expansion.

command:
  - /bin/ls
  - -l
  - /app

If you specify it as a simple string, it is implicitly wrapped in sh -c '...', and that shell will do variable expansion, which is what you want in your case.

command: microservice $VAR1 $VAR2

(Your form is not only not doing variable expansion, but because you specified the command in a single list item, it is looking for a file literally named microservice $VAR1 $VAR2, spaces and dollar signs included, to be the main container process.)

So I didn't quite get how you want me to change my command, could you explain a bit more? — HHH, Sep 03 '19 at 17:35

score 0 · Answer 3 · answered Sep 04 '19 at 12:28

Environment variables are most likely being set inside the container. However, the $ syntax is expanded by the compose file parser to inject settings from your shell on the host. To expand them inside the container, you need to escape them with the $$ syntax:

services:
  mlx-python-hdfs:
    image: image_name
    container_name: cname
    env_file: ./variables.txt
    command:
         - microservice $$VAR1 $$VAR2

That will pass a literal $ into the container which will be expanded by a shell inside the container.

See the compose file documentation for more details: https://docs.docker.com/compose/compose-file/#variable-substitution

Note that renaming the file to .env results in the variables being set inside docker-compose itself, not inside your container. That will also work if you do not escape your variables.

How to set environment variables in docker-compose

3 Answers3