Can't reach service of host from container

Question

On the host, there is a service

@server# netstat -ln | grep 3308
tcp6       0      0 :::3308                 :::*                    LISTEN

It can be reached from remote. The container is in a user-defined bridge network. The server IP address is 192.168.1.30

@localhost ~]# ifconfig
br-a54fd3b63acd: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.18.0.1  netmask 255.255.0.0  broadcast 172.18.255.255
        inet6 fe80::42:1eff:fecc:92e8  prefixlen 64  scopeid 0x20<link>
        ether 02:42:1e:cc:92:e8  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        inet6 fe80::42:37ff:fe9f:e4f1  prefixlen 64  scopeid 0x20<link>
        ether 02:42:37:9f:e4:f1  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 34  bytes 4018 (3.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.30  netmask 255.255.255.0  broadcast 192.168.1.255

And ping from container also works.

@33208c18aa61:~# ping -c 2 192.168.1.30
PING 192.168.1.30 (192.168.1.30) 56(84) bytes of data.
64 bytes from 192.168.1.30: icmp_seq=1 ttl=64 time=0.120 ms
64 bytes from 192.168.1.30: icmp_seq=2 ttl=64 time=0.105 ms

And the service is available.

@server# telnet 192.168.1.30 3308
Trying 192.168.1.30...
Connected to 192.168.1.30.
Escape character is '^]'.
N

But the service can't be reached from the container.

@33208c18aa61:~# telnet 192.168.1.30 3308
Trying 192.168.1.30...
telnet: Unable to connect to remote host: No route to host

I checked Make docker use IPv4 for port binding make sure I didn't have IPv6 set to only bind on IPv6

# sysctl net.ipv6.bindv6only
net.ipv6.bindv6only = 0

From inside of a Docker container, how do I connect to the localhost of the machine? find my route is a little different.

# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         router.asus.com 0.0.0.0         UG    100    0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-a54fd3b63acd
192.168.1.0     0.0.0.0         255.255.255.0   U     100    0        0 eth0

Does it matter? Or could it be another reason?

Esteban Garcia · Answer 1 · 2019-04-18T10:59:59.257

Your docker container is on a different network namespace and connected to a different interface than your host machine that's why you can't reach it using the ip 192.168.x.x

What you need to do is to use the docker network gateway instead, in your case 172.17.0.1 but be aware that this IP might no be the same from host to host so to reproduce this everywhere and be completely sure of which is the IP you can create an user-defined network specifying the subnet and gateway and running your container there for example:

docker network create -d bridge --subnet 172.16.0.0/24 --gateway 172.16.0.1 dockernet
docker run --net=dockernet ubuntu

Also whatever service you are trying to connect here must be listening on the docker's bridge interface as well.

Another option is to run the container on the same network namespace as the host with the --net=host flag, and in this case you can access service outside the container using localhost

the `192.168.1.30` is the host machine. Why `--subnet 172.16.0.0/24` when the docker0 is `172.17.0.1`? It doesn't work here. — Shihe Zhang, Apr 19 '19 at 01:06
If the network is not correct, how could the container `ping` host success? — Shihe Zhang, Apr 19 '19 at 01:57

score 0 · Accepted Answer · answered Apr 19 '19 at 02:55

Inspired by the official document

The Docker bridge driver automatically installs rules in the host machine so that containers on different bridge networks cannot communicate directly with each other.

I checked the iptables on the server, for an experiment I stopped the iptables temporary. Then the container can reach that service success. Later I was told, the server has been reboot recently. So guessing some config was lost after that reboot. Not familiar with iptables very much, and when I try

systemctl status iptables.service

It says the service is not installed. After I install and run the service,

iptables -L -n

is almost empty. Now not clue what kind of iptables rules can cause that messy.
But if anyone face the ping success telnet fail situation, iptables could be the place of the root cause.

Can't reach service of host from container

2 Answers2