Log client's "real" IP address in Docker Swarm 1.12 when accessing a service

Question

I have nginx container running as a service in Docker Swarm inside user created overlay network. Both created with:

docker network create --driver overlay proxy
docker service create --name proxy --network proxy -p 80:80 nginx

When accessing nginx site through a browser, in nginx access log remote address is logged as 10.255... formatted address, what I presume to be the Swarm load balancer address. The question is how to know/log the address of the end client accessing the site and not the load balancer address.

Farhad Farahi · Answer 1 · 2017-02-27T07:52:57.830

10

Good catch!, Most people analyzing the nginx access.log and client ip is important part of it.

As docker version 1.12.1 the problem exists. nginx will log swarm overlay ip. But client ip logs fine as standalone container. As a work around, you can have a reverse proxy pointing to swarm service. I know this is against High availablity and Self Healing concept of swarm, but seems to be the only work around right now.

sample config: (lets assume swarm service is listening on 8081 on localhost)

server {
  listen 80 default_server;
  location / {
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_pass          http://localhost:8181;
    proxy_read_timeout  90;
  }
}

More info can be found on this github issue.

Another Option:

You can use networking in host mode.

docker service create \
--name nginx \
--network <your overlay network> \
--publish mode=host,target=80,published=80 \
--publish mode=host,target=443,published=443 \
--replicas 1 \
nginx

edited Feb 27 '17 at 07:52

answered Oct 06 '16 at 08:18

Farhad Farahi

35,528
7
73
70

I wanted to note that using this in a production environment is a security risk, especially if your application relies on these values to validate the client. The X-Real-IP and X-Forwarded headers can easily be spoofed and should not be relied upon. – tkeeler Jan 25 '17 at 23:29
@tkeeler with haproxy you can unset the the variables in the http request and the set real value in the response – c4f4t0r May 28 '17 at 07:25
@c4f4t0r - Only if you restrict all HTTP requests to haproxy. Otherwise someone can make a request directly to the server bypassing your haproxy rewrite. – tkeeler May 30 '17 at 00:04
@tkeeler I think the service need to be exposed to the world only using the lb and nobody can use one single server. – c4f4t0r May 30 '17 at 09:37

score 7 · Answer 2 · answered Feb 07 '17 at 10:24

7

So the way I've worked around this for now is by using the mode=host option for publishing the port and pinning the proxy container to a single node like so:

docker service create \
--name proxy \
--network proxy \
--publish mode=host,target=80,published=80 \
--publish mode=host,target=443,published=443 \
--constraint 'node.hostname == myproxynode' \
--replicas 1 \
letsnginx

answered Feb 07 '17 at 10:24

sas

7,017
4
36
50

1

`mode=host` option has been added in Docker 1.13 and is what I'm doing now too. – eskp Feb 08 '17 at 12:58

score 2 · Answer 3 · edited Feb 19 '21 at 01:25

We had to solve the issue of identifying clients real IPs for our production deployment of Docker Swarm running https://www.newsnow.co.uk/ (mode=host was not an option as we needed to run multiple replicas across multiple nodes).

To solve it, we created: https://github.com/newsnowlabs/docker-ingress-routing-daemon, a daemon that modifies the ingress mesh network routing to expose true client IPs to service containers.

Log client's "real" IP address in Docker Swarm 1.12 when accessing a service

3 Answers3

Linked