How to cache docker-compose build inside github-action

Question

Is there any way to cache docker-compose so that it will not build again and again? here is my action workflow file:

name: Github Action
on:
  push:
    branches:
      - staging
jobs:
  test:
    runs-on: ubuntu-18.04

    steps:
      - uses: actions/checkout@v1

      - name: Bootstrap app on Ubuntu
        uses: actions/setup-node@v1
        with:
          node-version: '12'


      - name: Install global packages
        run: npm install -g yarn prisma


      - name: Install project deps
        if: steps.cache-yarn.outputs.cache-hit != 'true'
        run: yarn


      - name: Build docker-compose
        run: docker-compose -f docker-compose.test.prisma.yml up --build -d

I want to cache the docker build step. I have tried using if: steps.cache-docker.outputs.cache-hit != 'true' then only build but didn't work.

DannyB · Accepted Answer · 2020-12-09T12:34:58.877

12

What you are referring to is called "docker layer caching", and it is not yet natively supported in GitHub Actions.

This is discussed extensively in several places, like:

As mentioned in the comments, there are some 3rd party actions that provide this functionality (like this one), but for such a core and fundamental feature, I would be cautious with anything that is not officially supported by GitHub itself.

edited Dec 09 '20 at 12:34

answered Apr 29 '20 at 14:46

DannyB

12,810
5
55
65

so, it is not possible to do with github action now? – Ashik Apr 29 '20 at 22:30
What about https://github.com/satackey/action-docker-layer-caching? – Lucas Bustamante Aug 27 '20 at 17:10
I tried that one, but it can't work with sha's for images. – Vaal Jan 18 '21 at 16:45
docker's build-push-action does not support layer caching for docker-compose directly: https://github.com/docker/build-push-action/issues/128 But there is a workaround that I haven't tested yet: https://github.com/docker/build-push-action/issues/150 – Scott McAllister Aug 30 '21 at 15:32

score 10 · Answer 2 · answered Jan 18 '21 at 16:47

For those arriving here via Google, this now "supported". Or at least it is working: https://github.community/t/use-docker-layer-caching-with-docker-compose-build-not-just-docker/156049. The idea is to build the images using docker (and its cache) and then use docker compose to run (up) them.

score 6 · Answer 3 · answered Feb 23 '23 at 11:20

This question is old, but I found myself trying to solve precisely the same problem. After reading many different answers and spending lots of time, I eventually found a decent solution.

My workflow file now looks like this:

jobs:
  build:
    name: Integration tests
    runs-on: ubuntu-22.04
    # I need "packages: write" to access GHCR.
    # More about permissions here: https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
    permissions: write-all
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Login to Docker Registry
        uses: docker/login-action@v2
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: actions/checkout@v2

      - name: Build docker-compose
        # Before the cache, it was "run: docker compose build".
        run: docker buildx bake --file docker-compose.yml --file docker-compose-cache.json

...

And in the docker-compose-cache.json file, I have the following:

{
  "target": {
    "service-name": {
      "cache-from": [
        "type=registry,ref=ghcr.io/MY_GITHUB_ORG_NAME/service-name:cache"
      ],
      "cache-to": [
        "type=registry,ref=ghcr.io/MY_GITHUB_ORG_NAME/service-name:cache"
      ],
      "output": [
        "type=docker"
      ]
    }
}

For each service in docker-compose.yml, I add a target in docker-compose-cache.json. docker buildx bake takes build instructions from docker-compose.yml and cache instructions from docker-compose-cache.json.

This way, I can still use docker-compose up --build locally as usual.

As you may note, I'm using the GitHub container registry instead of the GitHub actions cache because GHCR has no limitation for cache size.

Perfect, exactly what I needed! This is the best answer. – Anthony Garcia-Labiad Jun 12 '23 at 13:57 — Anthony Garcia-Labiad, Jun 12 '23 at 13:57

score 2 · Answer 4 · answered Aug 19 '22 at 08:55

If using docker/bake-action or docker/build-push-action & want to access a cached image in subsequent steps -

Use load:true to save the image
Use the same image name as the cached image across steps in order to skip rebuilds.

Example:

...
        name: Build and push
        uses: docker/bake-action@master
        with:
          push: false
          load: true
        set: |
            web.cache-from=type=gha
            web.cache-to=type=gha
      -
        name: Test via compose
        command: docker compose run web tests
...

services:
  web:
    build:
      context: .
    image: username/imagename
    command: echo "Test run successful!"

See the docker team's responses;

How to cache docker-compose build inside github-action

4 Answers4

Linked