Connect to multiple pre-existing external networks in docker

Question

I have two different pre-existing networks created by two separated docker-compose operations. The first one is Kafka and the second one is MySQL DB. The third container is a Kafka consumer, which has to have access to the DB and Kafka topic. So I expect to connect them in another docker-compose something like this so I could use services like "db" or "broker" defined somewhere external:

version: '2'
services:
  web:
    build: .
    environment:
      KAFKA_BROKER: broker:29092
      DB_MYSQL_USER: ${DB_MYSQL_REMOTE_USER}
      DB_MYSQL_PASS: ${DB_MYSQL_PASS}
      DB_MYSQL_ADDRESS: db:3306
      KAFKA_TOPIC_NAME: ${KAFKA_TOPIC_NAME}
      KAFKA_GROUP_NAME: ${KAFKA_GROUP_NAME}

networks:
  db_container_default:
    external: true
  kafka_client_default:
    external: true

Eventually, the container doesn't recognize these networks at all.

The only way to properly connect to ONLY ONE network is here.
So I ended up with settings like this, which is utter lame(connect to the exposed port of the DB by host-ip):

version: '2'
services:
  web:
    build: .
    environment:
      KAFKA_BROKER: broker:29092
      DB_MYSQL_USER: ${DB_MYSQL_REMOTE_USER}
      DB_MYSQL_PASS: ${DB_MYSQL_PASS}
      DB_MYSQL_ADDRESS: 192.168.1.103:3307
      KAFKA_TOPIC_NAME: ${KAFKA_TOPIC_NAME}
      KAFKA_GROUP_NAME: ${KAFKA_GROUP_NAME}

networks:
  default:
    external:
      name: kafka_client_default

How should I properly configure my third docker-compose?

Like in https://stackoverflow.com/questions/38088279/communication-between-multiple-docker-compose-projects. Maybe you can chain all three of them. i.e put all of them in the same network. That way each container can access each others by service name. — Harsh Verma, Jan 08 '21 at 11:39

score 1 · Accepted Answer · answered Jan 08 '21 at 12:04

You have to actually declare that the containers should be attached to the networks.

version: '2'
services:
  web:
    # Each service needs to declare what networks it attaches to
    # The default is to only attach to "default"
    # If anything is explicitly listed then "default" is not used
    networks:
      - db_container_default
      - kafka_client_default

# You also need to declare at the top level that the networks exist
networks:
  db_container_default:
    external: true
  kafka_client_default:
    external: true

You might be able to simplify this by having fewer Compose files (for example, declare the application's database in the same docker-compose.yml as the application itself), or by configuring all of the parts to have the same default network.

Connect to multiple pre-existing external networks in docker

1 Answers1