How to run project with Gitlab CI registry variables?

Question

Can you explain. I have Gitlab runner and now I am configuring CI/CD using one guide. It says that runner requires the image names in docker-compose.yml to be named like this:

$CI_REGISTRY/organisation/path-to-project/project_image:$CI_ENVIRONMENT_SLUG-$CI_COMMIT_SHA

I put this value in a variable in the .env file. But now when I run docker compose up - error pops up - says $CI_REGISTRY, $CI_ENVIRONMENT_SLUG and $CI_COMMIT_SHA are not set. What should I do in this case? Everything was working fine before CI/CD was connected.

level=warning msg="The \"CI_REGISTRY\" variable is not set. Defaulting to a blank string."       
level=warning msg="The \"CI_ENVIRONMENT_SLUG\" variable is not set. Defaulting to a blank string."
level=warning msg="The \"CI_COMMIT_SHA\" variable is not set. Defaulting to a blank string."     
Error response from daemon: no such image: organisation/path-to-project/project_image:-: invalid reference format

At the same time docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY in before_script in .gitlab-ci.yml works and variable values are calculated from somewhere.

image: docker/compose:alpine-1.28.0

stages:
  - build
  - deploy

before_script:
  - apk add make
  - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  - cp $ENV $(pwd)/src/.env

build:
  stage: build
  script:
    - make build push
  tags:
    - job:build
  only:
    - main
  environment:
    name: production

deploy:
  stage: deploy
  script:
    - make pull down migrate up
  tags:
    - job:deploy
  only:
    - main
  environment:
    name: production

$ENV in before_script is variable on Gitlab.

Here is docker-compose.yml (${IMAGE_NAME} - variable from .env):

services:
  app1:
    build: .
    image: ${IMAGE_NAME}
    env_file:
      - .env
    # ...

  app2:
    build: .
    image: ${IMAGE_NAME}
    env_file:
      - .env
    # ...

Can you tell me what I'm doing wrong? I'm just getting started with CI/CD.

Answer 1

This is about Docker Compose ¹ .

The env_file option defines environment variables that will be available inside the container only ^2,3 .

If however, you want it to be interpreted during the Gitlab CI/CD execution of the before_script / build.script or deploy.script commands, you need to have a file named .env placed at the root next to your docker-compose.yml file unless you use the --env-file option in the CLI ⁴ .

Try this on your own shell first, let's setup an example composition with a .env file:

$ printf 'name: ${EXAMPLE}\n' > docker-compose.EXAMPLE.yml
$ printf 'EXAMPLE=c-to-the-i-to-the-c-to-the-d\n' > .env.EXAMPLE
$ docker compose --env-file .env.EXAMPLE --file docker-compose.EXAMPLE.yml config
name: c-to-the-i-to-the-c-to-the-d
services: {}

The same it works when the Gitlab runner execute any of the script commands.

NOTE: Docker Compose V1 vs. V2: You have not shown the concrete docker-compose(1) commands in your question. The --env-file command-line option may only be available with docker compose ... and not docker-compose .... If you're using the docker-compose command change to the docker command with the compose plugin (availabe as sub-command).

Gitlab CI/CD before_script Docker Registry and Docker Compose Configuration Example

Here an example based on your Gitlab CI/CD configurations' before_script with some notes, the numbers in the comments relate to the numbered list below the example:

before_script:
  - apk add make
  - echo "$CI_REGISTRY_PASSWORD" | docker login --password-stdin -u "$CI_REGISTRY_USER" -- "$CI_REGISTRY" # (1)
  - cp "$ENV" ./src/.env # (2)
  - docker compose --env-file ./src/.env --file docker-compose.yml config # (3)

1. Always quote variables. Handle the non-happy path (e.g. variables are unset) (here the story for docker login in gitlab ci/cd and variables)
2. Always quote variables, again, and no need for $(pwd) (which then should be quoted, again, as well: "$(pwd)"). Less is better, often and then more often in the long run.
3. Run docker compose config to see if your configuration is as it should.

$ make install-local-runner

As you're getting started with Gitlab CI/CD, bring it next to you if you have not yet: Instructions. Roughly 500MB in size, you have gitlab-runner exec etc. afterwards and can actually deal with all those issues before they even touch ground far away and much later (Villarriba comes to mind: make local && make party).

---

1. https://docs.docker.com/compose/
2. https://docs.docker.com/compose/compose-file/05-services/#env_file
3. https://github.com/docker/compose/issues/4189#issuecomment-263458253
4. https://docs.docker.com/compose/environment-variables/set-environment-variables/#substitute-with-an-env-file