1

I use OpenStack for a while to manage my applications. Now I want to transfer them to docker as container per app because docker is more lightweight and efficient.

The problem is almost every thing related to networking went wrong in runtime.

In my design, every application container should have a static IP address and I can use hosts file to locate the container network.

Here is my implementation. (the bash filename is docker_addnet.sh)

# Useages
# docker_addnet.sh container_name IP
# interface name: veth_(containername)
# gateway 172.17.42.1

if [ $# != 2 ]; then
    echo -e "ERROR! Wrong args"
    exit 1
fi

container_netmask=16
container_gw=172.17.42.1

container_name=$1
bridge_if=veth_`echo ${container_name} | cut -c 1-10`
container_ip=$2/${container_netmask}
container_id=`docker ps | grep $1 | awk '{print \$1}'`
pid=`docker inspect -f '{{.State.Pid}}' ${container_name}`

echo "Contaner: " $container_name "pid: " $pid

mkdir -p /var/run/netns
ln -s /proc/$pid/ns/net /var/run/netns/$pid

brctl delif docker0 $bridge_if

ip link add A type veth peer name B
ip link set A name $bridge_if
brctl addif docker0 $bridge_if
ip link set $bridge_if up
ip link set B netns $pid
ip netns exec $pid ip link set dev B name eth0
ip netns exec $pid ip link set eth0 up
ip netns exec $pid ip addr add $container_ip dev eth0
ip netns exec $pid ip route add default via $container_gw

The script is use to set the static ip address of the container, then you run the container, you must append --net=none to manually setup the network

You can now start a container by

sudo docker run --rm -it --name repl --dns=8.8.8.8 --net=none clojure bash

and set the network by 

sudo zsh docker_addnet.sh repl 172.17.15.1

In the container bash, you can see the IP address by ip addr, the output is something like 

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
67: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 2e:7b:7e:5a:b5:d6 brd ff:ff:ff:ff:ff:ff
    inet 172.17.15.1/16 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::2c7b:7eff:fe5a:b5d6/64 scope link 
       valid_lft forever preferred_lft forever

So far so good.

Let's try to get the container host ip address by using clojure repl. First the repl by:

lein repl

next eval the code below 

(. java.net.InetAddress getLocalHost)

The clojure code is equals to 

System.out.println(Inet4Address.getLocalHost());

What you get is an exception

UnknownHostException 5a8efbf89c79: Name or service not known java.net.Inet6AddressImpl.lookupAllHostAddr (Inet6AddressImpl.java:-2)

Other things going weird is the RMI server cannot get client IP address by RemoteServer.getClientHost();.

So what may cause this issue? I remember that java sometimes get the wrong network configures, but I don't know the reason.

Shisoft
  • 4,197
  • 7
  • 44
  • 61

2 Answers2

7

The documentation for InetAddress.getLocalHost() says:

Returns the address of the local host. This is achieved by retrieving the name of the host from the system, then resolving that name into an InetAddress.

Since you didn't take any steps to make your static IP address resolvable inside the container, it doesn't work.

To find the address in Java without going via hostname you could enumerate all network interfaces via NetworkInterface.getNetworkInterfaces() then iterate over each interface inspecting each address to find the one you want. Example code at Getting the IP address of the current machine using Java

Another option would be to use Docker's --add-host and --hostname options on the docker run command to put in a mapping for the address you want, then getLocalHost() should work as you expect.

Community
  • 1
  • 1
Bryan
  • 11,398
  • 3
  • 53
  • 78
3

In my design, every application container should have a static IP address and I can use hosts file to locate the container network.

Why not rethink your original design? Let's start with two obvious options:

  1. Container linking
  2. Add a service discovery component

Container linking

This approach is described in the Docker documentation.

https://docs.docker.com/userguide/dockerlinks/

When launching a container you specify that it is linked to another. This results in environment variables being injected into the linked container containing the IP address and port number details of the collaborating container.

Your application's configuration stops using hard coded IP addresses and instead uses soft code references that are set at run-time.

Currently docker container linking is limited to a single host, but I expect this concept will continue to evolve into multi-host implementations. Worst case you could inject environment variables into your container at run-time.

Service discovery

This is a common approach taken by large distributed applications. Examples implementations of such systems would be:

With such a system in place, your back-end service components (eg database) would register themselves on startup and client processes would dynamically discover their location at run-time. This form of decoupled operation is very Docker friendly and scales very very well.

Mark O'Connor
  • 76,015
  • 10
  • 139
  • 185