docker compose failing on gitlab-ci build stage

Question

I am trying to build gitlab-ci but one of the stages is failing the build. I get stuck on build stage. it does not recognise python and i am trying to install it so i can build the image and get it tested with robot framework

gitlab-ci.yaml

image: python:latest

services:
  - name: docker:dind
    entrypoint: ["env", "-u", "DOCKER_HOST"]
    command: ["dockerd-entrypoint.sh"]

stages:
  - compile
  - build
  - test
  - deploy

variables:
  DOCKER_HOST: tcp://docker:2375
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ""
  MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt
  REPOSITORY_URL: $AWS_ACCOUNT_ID.dkr.ecr.eu-west-2.amazonaws.com/apps_web
  TASK_DEFINITION_NAME: apps_8000
  CLUSTER_NAME: QA-2
  SERVICE_NAME: apps_demo
  ARTIFACT_REPORT_PATH: "app/reports/"

before_script:
  - docker info
  - export IMAGE=$CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME
  - export WEB_IMAGE=$IMAGE:web
  - apk add --no-cache openssh-client bash
  - chmod +x ./setup_env.sh
  - bash ./setup_env.sh
  - docker login -u $CI_REGISTRY_USER -p $CI_JOB_TOKEN $CI_REGISTRY

unittests:
  stage: test
  before_script:
    - python -m venv env
    - source env/bin/activate
    - python -m pip install --upgrade pip
    - pip install -r app/app-requirements.txt
  variables:
    DOCKER_IMAGE_TAG: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}
  image: ${DOCKER_IMAGE_TAG}
  script:
    - source env/bin/activate
    - python app/manage.py jenkins --enable-coverage
  artifacts:
    reports:
      junit: app/reports/junit.xml
    paths:
      - $ARTIFACT_REPORT_PATH
    expire_in: 30 days
    when: on_success
  only:
    refs:
      - merge_requests
    variables:
      - $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "qa"

migrations:
  stage: compile
  before_script:
    - python -m venv env
    - source env/bin/activate
    - pip install -r app/app-requirements.txt
  script:
    - python app/manage.py makemigrations
  artifacts:
    paths:
      - "app/*/migrations/*.py"
    expire_in: 1 day
  only:
    refs:
      - merge_requests
    variables:
      - $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "qa"

build:
  image:
    name: docker/compose:1.25.4
    entrypoint: [ "" ]
  stage: build
  variables:
    DOCKER_IMAGE_TAG: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}
  before_script:
    - apt-get install python3
    - python -m venv env
    - source env/bin/activate
    - python -m pip install --upgrade pip
    - pip install -r app/app-requirements.txt
    - export IMAGE=$CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME
    - export WEB_IMAGE=$IMAGE:web
  script:
    - apk add --no-cache bash
    - chmod +x ./setup_env.sh
    - bash ./setup_env.sh
    - docker login -u $CI_REGISTRY_USER -p $CI_JOB_TOKEN $CI_REGISTRY
    - docker pull $IMAGE:web || true
    - docker-compose -f docker-compose.ci.yml build
    - docker push $IMAGE:web
    - docker tag app
    - docker build -t ${DOCKER_IMAGE_TAG} .
  after_script:
    - docker push ${DOCKER_IMAGE_TAG}
    - docker logout
  dependencies:
    - migrations
  only:
    refs:
      - merge_requests
    variables:
      - $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "qa"

deploy_qa:
  stage: deploy
  image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-ecs:latest
  before_script:
    - export IMAGE=$CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME
    - export WEB_IMAGE=$IMAGE:web
    - docker login -u $CI_REGISTRY_USER -p $CI_JOB_TOKEN $CI_REGISTRY
  script:
    - echo $IMAGE
    - echo $WEB_IMAGE
    - docker pull $WEB_IMAGE
  environment:
    name: qa
    url: https://app.domain.com
  only:
    refs:
      - merge_requests
    variables:
      - $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "qa"

It is failing with error /bin/sh: eval: line 153: apt-get: not found

Answer 1

Like @slauth said in his comment, the docker/compose image is based on Alpine Linux which uses the apk package manager, not apt. However, you most likely wouldn't be able to use a debian image since you need the functionality of docker/compose. In that case, you can use apk to install python instead of apt-get, just like you're installing bash in the script section of this job: apk add --update --no-cache python3 && ln -sf python3 /usr/bin/python (This comes from a related answer here).

However, Installing and updating packages in a CI/CD Pipeline is generally a bad practice since depending on the number of pipelines you run, it can significantly slow down your development process. Instead, you can create your own docker images based on whichever image you need, and install your packages there. For example, you can create a new image based on docker/composer, install python, bash, etc there. Then push the new image either to Docker Hub, Gitlab's built-in docker registry, or another registry you might have available. Finally, in your .gitlab-ci.yml file, you simply change docker/compose to your new image.

For more information on this part, you can see another answer I wrote for a similar question here.