3

I need to get client IP in Kubernetes, but I can only get the Flannel internal IP 10.244.X.1 (in nginx via $remote_addr, $http_x_forwarded_for is always -).

The Network topo is:
client browser -> k8s cluster node ip -> k8s service (A) -> the pod (which desire the client ip) -> other services

I have tried externalTrafficPolicy in A, doesn't work. Is it only work on cloud provider, or behind a LB? [1]
I also tried ingress-nginx [2], get the same result.

My Environment:

 Kernel Version:             3.10.0-1062.4.1.el7.x86_64
 OS Image:                   CentOS Linux 7 (Core)
 Operating System:           linux
 Architecture:               amd64
 Container Runtime Version:  docker://18.6.2
 Kubelet Version:            v1.16.2
 Kube-Proxy Version:         v1.16.2
 Network Add-on:             Flannel

I have read some similar questions [3] [4] [5] , looks answers are imply to cloud provider or LB.

atealxt
  • 186
  • 1
  • 6
  • Maybe give Inlets a try, see https://blog.alexellis.io/ingress-for-your-local-kubernetes-cluster/ – Michael Hausenblas Dec 26 '19 at 05:35
  • @MichaelHausenblas looks it still require additional provider/node service, and access from public internet to private cluster is a different scenario, which is not what I asked. – atealxt Dec 27 '19 at 15:53

1 Answers1

1

Kubernetes services work in one of three proxy modes: Userspace, iptables, and IPVS. I'm not familiar with IPVS, but in both Userspace and iptables modes, the client IP will be altered for NodePort and Load Balancer services [docs].

Due to this proxying behaviour, the client IP will necessarily change, and that data can only be preserved if the client and server are speaking a "higher layer" protocol and there's a "higher layer" proxy that can capture the original client IP and inject it into the request before forwarding it on. If client and server are talking HTTP, then there are standard headers that an IaaS L7/HTTP(S) load balancer or nginx can inject for you, however if you're using nginx-ingress then the problem is that is sitting behind a NodePort or LoadBalancer service, so by the time it gets to the actual nginx process the client IP has been altered. You would need to have nginx running outside the cluster to make this work.

If you're running this on a public cloud, I think most cloud's L7 LB solutions inject these X-Forwarded-For or Forwarded headers out of the box for you. Alternatively, consider designing your server such that it doesn't need the client IP -- if client IP is being used to authenticate a client or authorize a request, there are other more cloud-native patterns that could work instead.

Amit Kumar Gupta
  • 17,184
  • 7
  • 46
  • 64
  • [This blog](https://www.haproxy.com/blog/preserve-source-ip-address-despite-reverse-proxies/) by HAProxy reveals it's possible in L3/4 layer, not sure if K8S have this or alternate. – atealxt Dec 27 '19 at 15:45