How to setup Gitlab CI E2E tests using Multiple dockers

Question

I am a bit lost with the automated testing using Gitlab CI. I hope I can explain my problem so somebody can help me. I'll try to explain the situation first, after which I'll try to ask a question (which is harder than it sounds)

Situation

Architecture

1. React frontend with Jest unit tests and Cypress e2e tests
2. Django API server 1 including a Postgres database and tests
3. Django API server 2 with a MongoDB database (which communicates with the other API

Gitlab

For the 2 API's, there is a Docker and a docker-compose file. These work fine and are set up correctly.

We are using GitLab for the CI/CD, there we have the following stages in this order:

1. build: where dockers for 1, 2 & 3 are build separate and pushed to private-registry
2. Test: Where the unit testing and e2e test (should) run
3. Release: where the docker images are released
4. Deploy: Where the docker images are deployed

Goal

I want to set up the GitLab CI such that it runs the cypress tests. But for this, all build dockers are needed. Currently, I am not able to use all dockers together when performing the end-to-end tests.

Problem

I don't really get how I would achieve this.

• Can I use the dockers that are built in the build stage for my e2e tests and can somebody give me an example of how this would be achieved? (By running the build docker containers as a service?)
• Do I need one Docker-compose file including all dockers and databases?
• Do I even need a dind?

I hope somebody can give me some advice on how to achieve this. An example would be even better but I don't know if somebody would want to do that.

Thanks for taking the time to read!

(if needed) Example of the API server 1

build-api:
  image: docker:19
  stage: build
  services:
    - docker:19-dind
  script:
    cd api
    docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    docker pull $IMAGE_TAG_API:latest || true
    docker build -f ./Dockerfile --cache-from $IMAGE_TAG_API:latest --tag $IMAGE_TAG_API:$CI_COMMIT_SHA .
    docker push $IMAGE_TAG_API:$CI_COMMIT_SHA

test-api:
  image: docker:19
  stage: test
  services:
    - postgres:12.2-alpine
    - docker:19-dind
  variables:
    DB_NAME: project_ci_test
    POSTGRES_HOST_AUTH_METHOD: trust
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker pull $IMAGE_TAG_API:$CI_COMMIT_SHA
    - docker run $IMAGE_TAG_API:$CI_COMMIT_SHA sh -c "python manage.py test"
  after_script:
    - echo "Pytest tests complete"
  coverage: "/TOTAL.+ ([0-9]{1,3}%)/"

release-api-staging:
  image: docker:19
  stage: release
  services:
    - docker:19-dind
  only:
    refs: [ master ]
    changes: [ ".gitlab-ci.yml", "api/**/*" ]
  environment:
    name: staging
  script:
    - docker pull $IMAGE_TAG_API:$CI_COMMIT_SHA
    - docker tag $IMAGE_TAG_API:$CI_COMMIT_SHA $IMAGE_TAG_API:latest
    - docker push $IMAGE_TAG_API:latest

Answer 1

The answer is a bit late, but still i'll try to explain the approach briefly for other developers with same issues. I also created an example project, contain 3 microservices in GitLab, where Server A runs end-to-end tests and is dependend on Server B and Server C.

When e2e test full-stack applications you have to either:

• mock all the responses of the microservices
• test against a deployed environment;
• or spin-up the environment temporary in the pipeline

As you noted, you want to spin-up the environment temporary in the pipeline. The following steps should be taken:

1. Deploy all backends as docker images in GitLab's private registry;
2. Mimic your docker-compose.yml services in 1 job in the pipeline;
3. connect the dots together.

Deploy backends as docker images in GitLab private registry

First you have to publish your docker images in the private registry of GitLab. You do this, because you now can reuse those images in another job. For this approach you need docker:dind. A simple example job to publish to a private registry on gitlab looks like:

before_script:
  - echo -n $CI_JOB_TOKEN | docker login -u gitlab-ci-token --password-stdin $CI_REGISTRY

publish:image:docker:
  stage: publish
  image: docker
  services:
    - name: docker:dind
      alias: docker
  variables:
    CI_DOCKER_NAME: ${CI_REGISTRY_IMAGE}/my-docker-image
  script:
    - docker pull $CI_REGISTRY_IMAGE || true
    - docker build --pull --cache-from $CI_REGISTRY_IMAGE --tag $CI_DOCKER_NAME --file Dockerfile .
    - docker push $CI_DOCKER_NAME
  only:
    - master

To see a real-world example, I have an example project that is public available.

Mimic your docker-compose.yml services in 1 job in the pipeline

Once you dockerized all backends and published the images on a private registry, you can start to mimic your docker-compose.yml with a GitLab job. A basic example:

test:e2e:
   image: ubuntu:20.04
   stage: test
   services:
      - name: postgres:12-alpine
        alias: postgress
      - name: mongo
        alias: mongo
      # my backend image
      - name: registry.gitlab.com/[MY_GROUP]/my-docker-image
        alias: server
   script:
      - curl http://server:3000 # expecting server exposes on port 3000, this should work
      - curl http://mongo:270117 # should work
      - curl http://postgress:5432 # should work!

Run the tests

Now everything is running in a single job in GitLab, you can simply start your front-end in detached mode and run cypress to test it. Example:

 script:
   - npm run start & # start in detached mode
   - wait-on http://localhost:8080 # see: https://www.npmjs.com/package/wait-on
   - cypress run # make sure cypress is available as well

Conclusion

Your docker-compose.yml is not meant to run in a pipeline. Mimic it instead using GitLab services. Dockerize all backends and store them in GitLab's private registry. Spin up all services in your pipeline and run your tests.

Answer 2

This article might shed some light. https://jessie.codes/article/running-cypress-gitlab-ci/ Essentially, you make two docker composers, one for your Cypress test and one for your items that are to be tested. This gets around the issues with images being able to access node and docker.