2

Let me first explain what I'm trying to do, as there may be multiple ways to solve this. I have two containers in docker 1.9.0:

  • node001 (172.17.0.2) (sudo docker run --net=<<bridge or test>> --name=node001 -h node001 --privileged -t -i -v /sys/fs/cgroup:/sys/fs/cgroup <<image>>)
  • node002 (172.17.0.3) (,,)

When I launch them with --net=bridge I get the correct value for SSH_CLIENT when I ssh from one to the other:

[root@node001 ~]# ssh root@172.17.0.3 root@172.17.0.3's password: [root@node002 ~]# env | grep SSH_CLIENT SSH_CLIENT=172.17.0.3 56194 22 [root@node001 ~]# ping -c 1 node002 ping: unknown host node002

In docker 1.8.3 I could also use the hostnames I supply when I start them, in 1.8.3 that last ping statement works!

In docker 1.9.0 I don't see anything being added in /etc/hosts, and the ping statement fails. This is a problem for me. So I tried creating a custom network...

docker network create --driver bridge test

When I launch the two containers with --net=test I get a different value for SSH_CLIENT:

[root@node001 ~]# ssh root@172.18.0.3 root@172.18.0.3's password: [root@node002 ~]# env | grep SSH_CLIENT SSH_CLIENT=172.18.0.1 57388 22 [root@node001 ~]# ping -c 1 node002 PING node002 (172.18.0.3) 56(84) bytes of data. 64 bytes from node002 (172.18.0.3): icmp_seq=1 ttl=64 time=0.041 ms

Note that the ip address is not node001's, it seems to represent the docker host itself. The hosts file is correct though, containing:

172.18.0.2 node001 172.18.0.2 node001.test 172.18.0.3 node002 172.18.0.3 node002.test

My current workaround is using docker 1.8.3 with the default bridge network, but I want this to work with future docker versions.

  • Is there any way I can customize the test network to make it behave similarly to the default bridge network?

Alternatively:

  • Maybe make the default bridge network write out the /etc/hosts file in docker 1.9.0?

Any help or pointers towards different solutions will be greatly appreciated..

Edit: 21-01-2016

Apparently the problem is fixed in 1.9.1, with bridge in docker 1.8 and with a custom (--net=test) in 1.9.1, now the behaviour is correct:

[root@node001 tmp]# ip route default via 172.17.0.1 dev eth0 172.17.0.0/16 dev eth0 proto kernel scope link src 172.17.0.5

[root@node002 ~]# env | grep SSH_CLIENT SSH_CLIENT=172.18.0.3 52162 22

Retried in 1.9.0 to see if I wasn't crazy, and yeah there the problem occurs:

[root@node001 tmp]# ip route default via 172.18.0.1 dev eth0 172.18.0.0/16 dev eth0 proto kernel scope link src 172.18.0.3

[root@node002 ~]# env|grep SSH_CLI SSH_CLIENT=172.18.0.1 53734 22

So after remove/stop/start-ing the instances the IP-addresses were not exactly the same, but it can be easily seen that the ssh_client source ip is not correct in the last code block. Thanks @sourcejedi for making me re-check.

Ray Burgemeestre
  • 1,230
  • 8
  • 13

1 Answers1

1

Firstly, I don't think it's possible to change any settings on the default network, i.e. to write /etc/hosts. You apparently can't delete the default networks, so you can't recreate them with different options.

Secondly

Docker is careful that its host-wide iptables rules fully expose containers to each other’s raw IP addresses, so connections from one container to another should always appear to be originating from the first container’s own IP address. docs.docker.com

I tried reproducing your issue with the random containers I've been playing with. Running wireshark on the bridge interface for the network, I didn't see my ping packets. From this I conclude my containers are indeed talking directly to each other; the host was not doing routing and NAT.

You need to check the routes on your client container ip route. Do you have a route for 172.18.0.2/16? If you only have a default route, it could try to send everything through the docker host. And it might get confused and do masquerading as if it was talking with the outside world.

This might happen if you're running some network configuration in your privileged container. I don't know what's happening if you're just booting it with bash though.

sourcejedi
  • 3,051
  • 2
  • 24
  • 42