GitLab: Mount /builds in build image as tmpfs

Question

In our GitLab CI environment we have a build server with lots of RAM but mechanical disks, running npm install takes a long time (I have added cache but it still needs to chew through existing packages so cache cannot solve all of this alone).

I want to mount /builds in the builder docker image as tmpfs but I'm having a hard time figuring out where to put this configuration. Can I do that in the builder image itself or maybye in .gitlab-ci.yml for each project?

Currently my gitlab-ci.yml looks like this:

image: docker:latest

services:
  - docker:dind

variables:
  DOCKER_DRIVER: overlay

cache:
  key: node_modules-${CI_COMMIT_REF_SLUG}
  paths:
    - node_modules/

stages:
  - test

test:
  image: docker-builder-javascript
  stage: test
  before_script:
    - npm install
  script:
    - npm test

score 3 · Accepted Answer · answered Nov 13 '18 at 14:33

I figured out that this was possible to solve by using the mount command straight in the before_script section, although this requires you to copy the source code over I managed to reduce the test time quite a lot.

image: docker:latest

services:
  - docker:dind

variables:
  DOCKER_DRIVER: overlay

stages:
  - test

test:
  image: docker-builder-javascript
  stage: test

  before_script:
    # Mount RAM filesystem to speed up build
    - mkdir /rambuild
    - mount -t tmpfs -o size=1G tmpfs /rambuild
    - rsync -r --filter=":- .gitignore" . /rambuild
    - cd /rambuild

    # Print Node.js npm versions
    - node --version
    - npm --version

    # Install dependencies
    - npm ci

  script:
    - npm test

Since I'm now using the npm ci command instead of npm install I have no use of the cache anymore since it's clearing the cache at each run anyway.

Which GitLab Runner Executor are you using? I didn't manage to make this work with the docker executor, which makes sense, since containers are usually unprivileged and can't mount anything. — jlh, Sep 16 '21 at 12:21

score 1 · Answer 2 · answered Oct 31 '18 at 14:13

You probably want something like this to add a data volume on the runner:

volumes = ["/path/to/volume/in/container"]

https://docs.gitlab.com/runner/configuration/advanced-configuration.html#example-1-adding-a-data-volume

I'd probably use the second option from the article though, and add the data volume from a host container, in case your cache gets corrupted for some reason, since it will be easier to clean.

volumes = ["/path/to/bind/from/host:/path/to/bind/in/container:rw"]

I've done this for a composer cache before, and it works very well. You should be able to set the cache for your npm using the following environment variable in your .gitlab-ci.yaml:

npm_config_cache=/path/to/cache

The other option is to use artifacts between builds, as outlined here: How do I mount a volume in a docker container in .gitlab-ci.yml?

Thanks! The cache itself works fine it's just that I want all of the build to run on a tmpfs filesystem since the disk I/O when checking the npm packages as well as JS code transpilation takes time. But of course if I mount an external directory I can have that directory setup as a tmpfs system, I was hoping though that I could do this in a way where I don't have to make settings for this on the host machine (I can if I have to but it makes it harder to re-install/migrate/duplicate that machine). — tirithen, Oct 31 '18 at 14:48

GitLab: Mount /builds in build image as tmpfs

2 Answers2