0

I'm trying to build an application that is able to use local integration testing via Docker Compose with Google Cloud emulator containers, while also being able to run that same Docker Compose configuration on a Docker-based CI/CD tool (Google Cloud Build).

The kind of docker-compose.yml configuration I'm using is:

version: '3.7'
services:
  
  main-application:
    build:
      context: .
      target: dev
    image: main-app-dev
    container_name: main-app-dev
    network_mode: $DOCKER_NETWORK
    environment:
      - MY_ENV=my_env
    command: ["sh", "-c", "PYTHONPATH=./ python app/main.py"]
    volumes:
      - ~/.config:/home/appuser/.config
      - ./app:/home/appuser/app
      - ./tests:/home/appuser/tests
    depends_on:
      - firestore

  firestore:
    image: google/cloud-sdk
    container_name: firestore
    network_mode: $DOCKER_NETWORK
    environment:
      - GCP_PROJECT_ID=dummy-project
    command: ["sh", "-c", "gcloud beta emulators firestore start --project=$$GCP_PROJECT_ID --host-port=0.0.0.0:9000"]
    ports:
      - "9000:9000"

I added the network_mode arguments to enable the configuration to use the "cloudbuild" network type available on the CI/CD pipeline, which is currently working perfectly. However this network configuration is not available to local Docker, hence why I've tried to use environment variables to enable the switch depending on local vs Cloud Build CI/CD environment.

Before I added these network_mode params/args for the CI/CD, the local testing was working just fine. However since I added them, my application either can't run, or can't connect to its accompanying services, like the firestore one specified in the YAML above. I have tried the following valid Docker network modes with no success:

  • "bridge" - runs the service, but doesn't allow connection between containers
  • "host" - doesn't allow the service to run because of not being compatible with assigning ports
  • "none" - doesn't allow the service to connect externally
  • "service" - doesn't allow the service to run due to invalid mode/service

Anyone able to provide advice on what I'm missing here? I would assume one of these network modes would be what Docker Compose would be using if the network_mode is not assigned, so I'm not sure why all of them are failing.

I want to avoid having a separate cloud build file for the remote and local configurations, and would also like to avoid the hassle of setting up my own docker network locally. Ideally if there were some way of only applying network_mode only remotely, that would work best in my case I think.

TL;DR: Specifying network_mode does not give me the same result as not specifying it when running docker-compose up locally. Due to running the configuration in the cloud I can't avoid specifying it.

Dom.MB
  • 73
  • 1
  • 6
  • What's the actual problem you're having? I'm not specifically familiar with `network_mode: cloudbuild` but I can tell you that networking between containers definitely works with `network_mode: bridge` (usually the default); see [Networking in Compose](https://docs.docker.com/compose/networking/) in the Docker documentation. (And remember that the cloud environment won't be able to use your local files, so I'd suggest removing the `volumes:` block so you're actually running the code in your image.) – David Maze Jul 31 '22 at 17:16
  • I'll update the question with a tl;dr, but basically the problem is specifying the network mode when running locally gives me different/bad results to running without specifying network mode. – Dom.MB Aug 01 '22 at 08:04

1 Answers1

0

Found a solution thanks to this thread and the comment by David Maze.

As far as I understand it, Docker Compose when not provided a specific network_mode for all the containers, creates its own private default network, named after the folder in which the docker-compose.yml file exists (as default).

Specifying a network mode like the default "bridge" network, without using this custom network created by docker compose means container discovery between services isn't possible, as in main-application couldn't find the firestore:9000 container.

Basically all I had to do was set the network_mode variable to myapplication_default, if the folder where docker-compose.yml sat in was called "MyApplication", to force app apps to use the same custom network set up in docker-compose up

Dom.MB
  • 73
  • 1
  • 6