1

I'm working on gitlab CI at my project, and I created image to make my tests and builds. When I ran it in docker executor every job needs to download images from beggining. Is there any way to cachee/store that layers at host? like cache? I tried store /var/lib/docker/aufs but I guest its not everythink. Can't find any solution anywhere. Anyone has that problem before? How to work with it?

gitlab-ci.yml

phpunit: stage: test script: - docker run -w /var/www -v $(pwd):/var/custom --rm phpunit/phpunit:4.8.5

gitlab-ci.yml with volumes from

phpunit: stage: test script: - docker run -w /var/www -v $(pwd):/var/custom --rm --volumes-from {image_from other build} phpunit/phpunit:4.8.5

Paweł Babilas
  • 35
  • 5
  • Did you try putting this line: `volumes = ["/var/lib/docker"]` in the `[runners.docker]` section of gitlab-ci-runner's `config.toml` file? Would also need automatic caching disabled using `disable_cache = false` – raarts Aug 14 '17 at 19:27
  • Yes I did it and after 2 hours my runner was out of space. Becouse every job was caching all the layers but also volumes... – Paweł Babilas Aug 16 '17 at 06:34
  • I can't test it myself right now, but what about mounting a volume like this: `volumes = ["/data/runner/:/var/lib/docker:rw"]`. Obviously only one runner per host allowed. – raarts Aug 16 '17 at 13:35

2 Answers2

0

The option that works best would be to use a dedicated long-running docker:dind container like this:

docker run --privileged --name gitlab-dind -d --restart=always  docker:dind

(add --storage-driver=overlay2 if your host supports it!)

Then modify config.toml as follows:

[runners.docker]
  tls_verify = false
  image = "docker:latest"   <--------
  privileged = false     <--------
  disable_cache = false
  volumes = ["/cache"]
  links = ["gitlab-dind:docker"]   <-----------
  shm_size = 0
[runners.cache]

And remove 

services:
- docker:dind

from .gitlab-ci.yml

If you want to run a docker container from .gitlab-ci.yml, you can do it like this:

test:
  stage: test
  image: phpunit/phpunit:4.8.5
  script:
  - cd your_source && phpunit

Note I am guessing about the last line, since I don't know phpunit.

raarts
  • 2,711
  • 4
  • 25
  • 45
  • Yes it would be great but I can not mount dir when Im doing docker run in container., it looks like that: docker run -w /var/www -v $(pwd):/var/custom --volumes-from=trunk_$CI_PIPELINE_ID composer/composer run-script post-install-cmd -- skip-application skip-config skip-upgrade skip-assets But custom is empty then :( – Paweł Babilas Aug 17 '17 at 09:19
  • I tried mount host dir to gitlab-dind and add it to config toml but still no success. Maybe there are some ready to use solutions? – Paweł Babilas Aug 17 '17 at 14:38
  • More information is needed. Please expand your question with more detail, including the .gitlab-di.yml file you are using. The solution I have given definitely should re-use images during a docker build. I have it running. – raarts Aug 17 '17 at 15:36
  • What I need: - I should store and reuse images - your solution solve this problem - when my script is run I want to run docker image with mounted dir from container, but when I do that my container has empty dir, no files from container are in my image. `phpunit: stage: test script: - docker run -w /var/www -v $(pwd):/var/custom --rm phpunit/phpunit:4.8.5` – Paweł Babilas Aug 17 '17 at 17:07
  • I put part of gitlab-ci in my question. – Paweł Babilas Aug 17 '17 at 17:10
  • I extended my answer. Hope it helps. – raarts Aug 17 '17 at 18:36
  • Thanks I'll try this, but have one last case, again updated in main post. You see I want to run container with volume from another container ;> – Paweł Babilas Aug 17 '17 at 18:48
  • I suppose you want to run unit tests. Wouldn't it be better to run those from your source tree? If you run it the way I suggested, your git tree would be available inside the container without you needing to mount it. – raarts Aug 17 '17 at 18:54
  • Your solution has more problem for me, when I want to make something with my files, or maybe git or smth, gitlab has no credentials git etc. So I need to use my custom private image instead, but there I want to run phpunit or maybe i'm doing it wrong? – Paweł Babilas Aug 17 '17 at 18:57
  • You mean that while building you want to pull from an external repo? Because while building obviously your own repo is already cloned and available. – raarts Aug 17 '17 at 19:02
0

OK I tried get this around, by making my service with volume

FROM docker:dind RUN mkdir /gitlab-runner VOLUME /gitlab-runner

And when run gitlab ci docker executor I mount volumes from docker with command

gitlab-ci-multi-runner exec docker phpunit --docker-volumes-from "gitlab-dind" --docker-links "gitlab-dind:docker" --builds-dir "/gitlab-runner"

But dunno why my docker image and docker service are not sharing volume, any ide? Gitlab mout there somethink itself? or it has no connection besides --docker-volumes-from?

Paweł Babilas
  • 35
  • 5
  • 1
    You keep extending the original question with more questions. I think it would be better if you create a new stack overflow question, and - since you said my answer solved your problem - mark my answer as correct? – raarts Aug 31 '17 at 08:45