1

I'm trying to find the settings I need to add to my self-hosted gitlab-runner in order to achieve the following: In a repository containing a dir/script.py I'm running

services:
  - docker:dind

main:
  image: docker:latest
  script:
    - docker run -i -v $(pwd)/dir:/mnt python:3 ls -la /mnt

with a GitLab shared runner which shows that dir/script.py has successfully been mounted into the container and is present as /mnt/script.py.

If I use my self-hosted gitlab-runner the directory is empty. I tried adding --privileged to docker run in .gitlab.yml as well as in the runner configuration in /etc/gitlab-runner/config.toml.

I'm pretty sure the mounted directory is empty inside the container because the mount path refers to the path on the host (see also Docker in Docker cannot mount volume). However, there seems to be a way for GitLab's shared runners to deal with that and I want to find and adopt this way to my self-hosted runner.

I'm running gitlab-runner and providing Docker-in-Docker by running the systemd unit

[Service]
Type=simple
ExecStartPre=/usr/bin/docker network create --subnet 172.25.0.0/16 gitlab-runner-net
ExecStartPre=/usr/bin/docker run -d --name gitlab-dind --privileged --restart always --network gitlab-runner-net -v /var/lib/docker -e DOCKER_TLS_CERTDIR= docker:19.03-dind --storage-driver=overlay2
ExecStartPre=/usr/bin/docker pull gitlab/gitlab-runner:latest
ExecStart=/usr/bin/docker run --name gitlab-runner --restart always --network gitlab-runner-net --dns 8.8.8.8 -v /srv/gitlab-runner/config:/etc/gitlab-runner -e DOCKER_HOST=tcp://gitlab-dind:2375 gitlab/gitlab-runner:latest
ExecStop=/usr/bin/docker stop gitlab-runner
ExecStop=/usr/bin/docker stop gitlab-dind
ExecStop=/usr/bin/docker rm gitlab-runner
ExecStop=/usr/bin/docker rm gitlab-dind
ExecStop=/usr/bin/docker network rm gitlab-runner-net

(extract) and by adding

host = "tcp://gitlab-dind:2375"
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
extra_hosts = ["gitlab-dind:172.25.0.2"]

to every runner section of the config.toml. This has been inspired by https://medium.com/@tonywooster/docker-in-docker-in-gitlab-runners-220caeb708ca. The setup allows to use docker build inside .gitlab-ci.yml and share the Docker cache between builds.

How do I mount a volume in a docker container in .gitlab-ci.yml? suggests using artifacts which is not an option since I want to test this scenario.

Kalle Richter
  • 8,008
  • 26
  • 77
  • 177
  • 1
    Don't have an answer since I'd want to test it, but have you looked at the shared runner settings? https://docs.gitlab.com/ee/user/gitlab_com/#configtoml another consideration is that the medium article is from 2+ years ago so a lot has changed especially with dind 19.03+ so I recommend checking out the dind part of the runner docs: https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-workflow-with-docker-executor – Arty-chan Oct 20 '19 at 02:18
  • @Arty-chan Thanks for your useful input. I updated by dind container to 19.03.1 without any change in the result. I didn't find anything in the config.toml of shared runners that I'd say might explain why mounting directories on shared runners works flawlessly. https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-workflow-with-docker-executor is a neat workaround, but I'd rather use shared runners than adjust my code to work on the CI with my self-hosted runner. – Kalle Richter Oct 20 '19 at 07:09

0 Answers0