How to add scripts to image before running them in gitlab CI

Question

I am trying to run a CI job in gitlab, where the integration tests depend on postgresql.

In gitlab I've used the postgresql runner. The issue is that the integration tests require the extension uuid-ossp. I could run the SQL commands before each test to ensure the extension is applied, but I'd rather apply it once before running all the tests.

So I've used the image tag in the CI script to add a .sh file in the postgresql image in /docker-entrypoint-initdb.d/, and then try to run the integration tests with the same image. The problem is that it doesn't seem to apply the extension as the integration tests fail where the uuid functions are used -- function uuid_generate_v4() does not exist

prep-postgres:
  stage: setup-db
  image: postgres:12.2-alpine
  script:
    - echo "#!/bin/bash

      set -e


      psql \"$POSTGRES_DB\" -v --username \"$POSTGRES_USER\" <<-EOSQL

      create extension if not exists \"uuid-ossp\";

      EOSQL" > /docker-entrypoint-initdb.d/create-uuid-ossp-ext.sh
  artifacts:
    untracked: true

test-integration:
  stage: test
  services:
    - postgres:12.2-alpine
  variables:
    POSTGRES_DB: db_name
    POSTGRES_USER: postgres
  script:
    - go test ./... -v -race -tags integration

An alternate i was hoping that would work was

prep-postgres:
  stage: setup-db
  image: postgres:12.2-alpine
  script:
    - psql -d postgresql://postgres@localhost:5432/db_name -c "CREATE EXTENSION IF NOT EXISTS \"uuid-ossp\";"
  artifacts:
    untracked: true

But in this case the client is unable to connect to postgres (i imagine it's because i'm editing the image not running it?)

I must be missing something obvious, or is this even possible?

score 1 · Answer 1 · answered Mar 17 '20 at 07:17

In both case in the job prep-postgres, you make changes in a running container (from postgres:12.2-alpine image) but you don't save these changes, so test-integration job can't use them.

I advice you to build your own image using a Dockerfile and the entrypoint script for the Postgres Docker image. This answer from @Elton Stoneman could help.

After that, you can refer your previously built image as services: in the test-integration job and you will benefit from the created extension.

score 0 · Accepted Answer · answered Mar 17 '20 at 10:04

At the moment i've had to do something a little smelly and download postgres client before running the extension installation.

.prepare_db: &prepare_db |
  apt update \
  && apt install -y postgresql-client \
  && psql -d postgresql://postgres@localhost/db_name -c "CREATE EXTENSION IF NOT EXISTS \"uuid-ossp\";"

test-integration:
  stage: test
  services:
    - postgres:12.2-alpine
  variables:
    POSTGRES_DB: db_name
    POSTGRES_USER: postgres
  script:
    - *prepare_db
    - go test ./... -v -race -tags integration

This isn't perfect. I was hoping that there was a way to save the state of the docker image between stages but there doesn't seem to be that option. So the options seem to be either:

install it during the test-integration stage.
create a base image specifically for this purpose where the installation of the expansion has already been done.

I've gone with option 1 for now, but will reply if i find something more concise, easier to maintain and fast.

How to add scripts to image before running them in gitlab CI

2 Answers2