3

So I have a multicontainer project running on AWS ElasticBeanstalk (latest Docker platform).

The containers share the same bridge network since they need to be able to communicate with each other (ports 3000, 8080, 9090, 9100). But one of those containers needs to be able to communicate with the host too (ports 55113, 56113).

This is normally possible by just adding extra_hosts: "host.docker.internal:host-gateway" to that container and then using host.docker.internal instead of localhost inside the container, i.e. host.docker.internal:55113. I've successfully used it before on a Debian.

But this is not working on AWS ElasticBeanstalk, the connection is refused.

Am I doing something wrong? Is there any workaround?

AWS ElasticBeanstalk Platform:

Docker running on 64bit Amazon Linux 2/3.4.2

Host's Docker & Docker-Compose versions:

$ sudo docker --version

Docker version 20.10.4, build d3cb89e

$ sudo docker-compose --version

docker-compose version 1.29.2, build unknown

Project's docker-compose.yml: (showing only the relevant parts)

version: "3.9"

services:
  prometheus:
    image: prom/prometheus:v2.28.1
    container_name: prometheus
    ports:
      - 9090:9090
    extra_hosts:
      - host.docker.internal:host-gateway # <-------- EXTRA HOST

networks:
  default:
    name: monitoring

Container debug:

$ sudo docker exec -it prometheus cat /etc/hosts

127.0.0.1   localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.1  host.docker.internal # <-------- the mapping is present
172.25.0.4  f01ffc6b2a8a

$ sudo docker exec -it prometheus wget host.docker.internal

Connecting to host.docker.internal (172.17.0.1:80)
wget: can't connect to remote host (172.17.0.1): Connection refused

$ sudo docker exec -it prometheus wget host.docker.internal:55113  # <-------- the port I want to reach

Connecting to host.docker.internal:55113 (172.17.0.1:55113)
wget: can't connect to remote host (172.17.0.1): Connection refused

Host's iptables (configured by AWS and/or Docker, I didn't edit it):

$ sudo iptables -S

-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o br-a8a58f19c627 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-a8a58f19c627 -j DOCKER
-A FORWARD -i br-a8a58f19c627 ! -o br-a8a58f19c627 -j ACCEPT
-A FORWARD -i br-a8a58f19c627 -o br-a8a58f19c627 -j ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.25.0.2/32 ! -i br-a8a58f19c627 -o br-a8a58f19c627 -p tcp -m tcp --dport 8080 -j ACCEPT
-A DOCKER -d 172.25.0.3/32 ! -i br-a8a58f19c627 -o br-a8a58f19c627 -p tcp -m tcp --dport 9100 -j ACCEPT
-A DOCKER -d 172.25.0.4/32 ! -i br-a8a58f19c627 -o br-a8a58f19c627 -p tcp -m tcp --dport 9090 -j ACCEPT
-A DOCKER -d 172.25.0.5/32 ! -i br-a8a58f19c627 -o br-a8a58f19c627 -p tcp -m tcp --dport 3000 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i br-a8a58f19c627 ! -o br-a8a58f19c627 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o br-a8a58f19c627 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN

Host's open ports:

$ netstat -tuplen

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name
tcp        0      0 0.0.0.0:9090            0.0.0.0:*               LISTEN      0          200315     -
tcp        0      0 127.0.0.1:55113         0.0.0.0:*               LISTEN      1000       119886     -
tcp        0      0 0.0.0.0:9100            0.0.0.0:*               LISTEN      0          200280     -
tcp        0      0 127.0.0.1:22221         0.0.0.0:*               LISTEN      1001       197452     -
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      0          14121      -
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      0          201081     -
tcp        0      0 127.0.0.1:56113         0.0.0.0:*               LISTEN      1000       119890     -
tcp        0      0 127.0.0.1:38581         0.0.0.0:*               LISTEN      0          20325      -
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      0          18585      -
tcp        0      0 0.0.0.0:3000            0.0.0.0:*               LISTEN      0          201164     -
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      0          16235      -
tcp6       0      0 ::1:55113               :::*                    LISTEN      1000       119885     -
tcp6       0      0 :::111                  :::*                    LISTEN      0          14124      -
tcp6       0      0 ::1:56113               :::*                    LISTEN      1000       119889     -
tcp6       0      0 :::22                   :::*                    LISTEN      0          18594      -
udp        0      0 0.0.0.0:68              0.0.0.0:*                           0          15762      -
udp        0      0 0.0.0.0:111             0.0.0.0:*                           0          14119      -
udp        0      0 127.0.0.1:323           0.0.0.0:*                           0          14990      -
udp        0      0 0.0.0.0:979             0.0.0.0:*                           0          14120      -
udp6       0      0 fe80::ed:19ff:fe84::546 :::*                                0          16539      -
udp6       0      0 :::111                  :::*                                0          14122      -
udp6       0      0 ::1:323                 :::*                                0          14991      -
udp6       0      0 :::979                  :::*                                0          14123      -
GG.
  • 21,083
  • 14
  • 84
  • 130
  • For now, **I just switched to `network_mode: host`**. I still can access the other containers (e.g. `localhost:9100`) and can access the processes running on the host (e.g. `localhost:55113`). I lose the isolation but I guess it doesn't matter much on ElasticBeanstalk. Also, note that [`network_mode: host` doesn't work on mac](https://github.com/docker/for-mac/issues/2716)… So I created another `docker-compose.mac.yml` for running the project in dev, that uses a classic bridge network and `host.docker.internal` to communicate with the host. – GG. Jul 08 '21 at 17:54
  • 1
    did you manage to resolve this? I'm facing similar issue using ```-add-host=host.docker.internal:host-gateway``` in ElasticBeanstalk. I can make is work with network_mode: host like you but that had other issues. – mikethe Dec 02 '21 at 18:39
  • @mikethe no I'm still using `network_mode: host` as described in the above comment. To be honest I didn't look more into it since it worked fine for my use case. There is probably a cleaner solution requiring to modify the iptables but I don't have enough expertise on the topic. – GG. Dec 02 '21 at 19:38

1 Answers1

1

I'm new in the field.

Your logs shows that your port is listening on localhost (127.0.0.1:56113) instead of all IPs (eg 0.0.0.0:8080).

In my case the port forward rule on the host machine has to be modified to -L0.0.0.0:<lport>:<ip>:<rport> .

  • Yes, according to https://stackoverflow.com/questions/70006997/access-host-from-within-a-docker-container "Also docker.host.internal is not your host loopback interface. If service you are trying to reach listens only on localhost interface then there is no chance to reach it without doing some magic with iptables / firewall." – kaikun May 17 '22 at 10:28