Docker Compose - Changing the network mode to "host" results in error: Error response from daemon: failed to add interface veth701c890 to sandbox

Question

Following this question, I edited my gateway container to use the host network mode:

services:
  gateway:
  ...
  network_mode: "host"

and then the docker compose up -d gives me this:

Error response from daemon: failed to add interface veth701c890 to sandbox: error setting interface "veth701c890" IP to 172.26.0.11/16: cannot program address 172.26.0.11/16 in sandbox interface because it conflicts with existing route {Ifindex: 4 Dst: 172.26.0.0/16 Src: 172.26.0.1 Gw: Flags: [] Table: 254

I restarted the docker and even the server. No luck.

The docker-compose.yml looks like this (only the gateway container has published ports):

version: '3.4'

services:
  gateway:
    image: <ms-yarp>
    environment:
      - ASPNETCORE_URLS=https://+:443;http://+:80
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./tls/:/tls/
    networks:
      - mynet
    restart: on-failure

  orders:
    image: <registry>/orders
    environment:
      - ASPNETCORE_URLS=http://+:80
    networks:
      - mynet
    restart: on-failure

  users:
    image: <registry>/users
    environment:
      - ASPNETCORE_URLS=http://+:80
    networks:
      - mynet
    restart: on-failure

  smssender:
    image: <registry>/smssender
    environment:
      - ASPNETCORE_URLS=http://+:80
    networks:
      - mynet
    restart: on-failure

  logger:
    image: <registry>/logger
    environment:
      - ASPNETCORE_URLS=http://+:80
    networks:
      - mynet
    restart: on-failure

  notifications:
    image: <registry>/notifications
    environment:
      - ASPNETCORE_URLS=http://+:80
    networks:
      - mynet
    restart: on-failure

  cacheserver:
    image: <registry>/redis
    networks:
      - mynet
    restart: on-failure

  ...

networks:
  mynet:

Answer 1

You can't combine host networking with any other Docker networking option. At least some versions of Compose have given warnings if you combine network_mode: host with other networks: or ports: options.

The other thing host networking means in this particular setup is that the one container that's using it is "outside Docker" for purposes of connecting to other containers. It works exactly the same way a non-container process would. That means the other containers need to publish ports: to be reachable from the gateway, and in turn the gateway configuration needs to use localhost and the published port numbers to reach the other containers.

version: '3.8'
services:
  gateway:
    image: <ms-yarp>
    network_mode: host
  orders:
    image: <registry>/orders
    ports:
      - '8001:80'
    networks:
      - mynet

{
  "ReverseProxy": {
    "Clusters": {
      "cluster": {
        "Destinations": {
          "orders": {
            "Address": "http://localhost:8001"
          }
        }
      }
    }
  }
}

Answer 2

Something like this: (doesn't work with Docker Desktop on windows WSL2, at least I couldn't even run the nginx example that is here in the docs)

version: '3.4'

services:
  gateway:
    image: <ms-yarp>
    environment:
      - ASPNETCORE_URLS=https://+:443;http://+:80
    network_mode: host
    volumes:
      - ./tls/:/tls/
    restart: on-failure

  orders:
    image: <registry>/orders
    environment:
      - ASPNETCORE_URLS=http://+:80
    ports:
      - 8080:80  
    networks:
      - mynet      
    restart: on-failure

  users:
    image: <registry>/users
    environment:
      - ASPNETCORE_URLS=http://+:80
    ports:
      - 8081:80            
    networks:
      - mynet      
    restart: on-failure

  smssender:
    image: <registry>/smssender
    environment:
      - ASPNETCORE_URLS=http://+:80
    ports:
      - 8082:80                  
    networks:
      - mynet      
    restart: on-failure

  logger:
    image: <registry>/logger
    environment:
      - ASPNETCORE_URLS=http://+:80
    ports:
      - 8082:80                  
    networks:
      - mynet      
    restart: on-failure

  notifications:
    image: <registry>/notifications
    environment:
      - ASPNETCORE_URLS=http://+:80
    ports:
      - 8083:80                  
    networks:
      - mynet      
    restart: on-failure

  cacheserver:
    image: <registry>/redis
    restart: on-failure
    networks:
      - mynet    

Also in your gateway service configuration you will need to change the

http://orders:80 to http://localhost:8080

http://users:80 to http://localhost:8081 and so on

Also restrict ports on the docker host of 8080 to 8083 to be accessible only from localhost and not from the internet.

You could even put all the containers (except the gateway) to a different docker host that is accessible only from the docker host where the gateway container is running and change the config in gateway from http://orders:80 to http://otherdockerhost:80 and so on.

But for this docker compose will not be viable you will need to "manually" create the containers with docker run commands (or have 2 separate compose project one for the gateway and one for the rest of the services) so this is where more serious container orchestration tools are required like kubernetes (you could try docker swarm or nomad or any other container orchestrator, but these are not so popular so if you are new to both kubernetes and docker swarm or all the other you are better off with starting with kubernetes, you will reap the benefits in the long run for both this project and your personal carrier too)