Accessing shell environment variables from docker-compose?

Question

How do you access environment variables exported in Bash from inside docker-compose?

I'm essentially trying to do what's described in this answer but I don't want to define a .env file.

I just want to make a call like:

export TEST_NAME=test_widget_abc
docker-compose -f docker-compose.yml -p myproject up --build --exit-code-from myproject_1

and have it pass TEST_NAME to the command inside my Dockerfile, which runs a unittest suite like:

ENV TEST_NAME ${TEST_NAME}

CMD python manage.py test $TEST_NAME

My goal is to allow running my docker container to execute a specific unittest without having to rebuild the entire image, by simply pulling in the test name from the shell at container runtime. Otherwise, if no test name is given, the command will run all tests.

As I understand, you can define environment variables in a .env file and then reference them in your docker-compose.yml like:

version: "3.6"
services:
  app_test:
    build:
      args:
      - TEST_NAME=$TEST_NAME
      context: ..
      dockerfile: Dockerfile

but that doesn't pull from the shell.

How would you do this with docker-compose?

Answer 1

For the setup you describe, I'd docker-compose run a temporary container

export COMPOSE_PROJECT_NAME=myproject
docker-compose run app_test python manage.py test_widget_abc

This uses all of the setup from the docker-compose.yml file except the ports:, and it uses the command you provide instead of the Compose command: or Dockerfile CMD. It will honor depends_on: constraints to start related containers (you may need an entrypoint wrapper script to actually wait for them to be running).

If the test code is built into your "normal" image you may not even need special Compose setup to do this; just point docker-compose run at your existing application service definition without defining a dedicated service for the integration tests.

---

Since Compose does (simple) environment variable substitution you could also provide the per-execution command: in your Compose file

version: "3.6"
services:
  app_test:
    build: ..
    command: python manage.py $TEST_NAME # uses the host variable

Or, with the Dockerfile you have, pass through the host's environment variable; the CMD will run a shell to interpret the string when it starts up

version: "3.6"
services:
  app_test:
    build: ..
    environment:
      - TEST_NAME # without a specific value here passes through from the host

These would both work with the Dockerfile and Compose setup you show in the question.

Answer 2

Environment variables in your docker-compose.yaml will be substituted with values from the environment. For example, if I write:

version: "3"
services:
  app_test:
    image: docker.io/alpine:latest
    environment:
      TEST_NAME: ${TEST_NAME}
    command:
      - env

Then if I export TEST_NAME in my local environment:

$ export TEST_NAME=foo

And bring up the stack:

$ docker-compose up
Creating network "docker_default" with the default driver
Creating docker_app_test_1 ... done
Attaching to docker_app_test_1
app_test_1  | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
app_test_1  | HOSTNAME=be3c12e33290
app_test_1  | TEST_NAME=foo
app_test_1  | HOME=/root
docker_app_test_1 exited with code 0

I see that TEST_NAME inside the container has received the value from my local environment.

---

It looks like you're trying to pass the environment variable into your image build process, rather than passing it in at runtime. Even if that works once, it's not going to be useful, because docker-compose won't rebuild your image every time you run it, so whatever value was in TEST_NAME at the time the image was built is what you would see inside the container.

It's better to pass the environment into the container at run time.