Kubernetes services : request Assignment algorithm

Question

What is the logic algorithm that a kubernets service uses to assign requests to pods that it exposes? Can this algorithm be customized?

Thanks.

Answer 1

kube-proxy in userspace mode chooses a backend via a round-robin algorithm.

kube-proxy in iptables mode chooses a backend at random.

IPVS provides more options for balancing traffic to backend Pods; these are:rr: round-robin,lc: least connection (smallest number of open connections),dh: destination hashing,sh: source hashing,sed: shortest expected delay, nq: never queue

As mentioned here:- Service

For application level routing you would need to use a service mesh like istio ,envoy, kong.

Answer 2

You can use a component kube-proxy. What is it?

kube-proxy is a network proxy that runs on each node in your cluster, implementing part of the Kubernetes Service concept. kube-proxy maintains network rules on nodes. These network rules allow network communication to your Pods from network sessions inside or outside of your cluster. kube-proxy uses the operating system packet filtering layer if there is one and it's available. Otherwise, kube-proxy forwards the traffic itself.

But why use a proxy when there is a round-robin DNS algorithm? There are a few reasons for using proxying for Services:

• There is a long history of DNS implementations not respecting record TTLs, and caching the results of name lookups after they should have expired.
• Some apps do DNS lookups only once and cache the results indefinitely.
• Even if apps and libraries did proper re-resolution, the low or zero TTLs on the DNS records could impose a high load on DNS that then becomes difficult to manage.

kube-proxy has many modes:

• User space proxy mode - In the userspace mode, the iptables rule forwards to a local port where a go binary (kube-proxy) is listening for connections. The binary (running in userspace) terminates the connection, establishes a new connection to a backend for the service, and then forwards requests to the backend and responses back to the local process. An advantage of the userspace mode is that because the connections are created from an application, if the connection is refused, the application can retry to a different backend
• Iptables proxy mode - In iptables mode, the iptables rules are installed to directly forward packets that are destined for a service to a backend for the service. This is more efficient than moving the packets from the kernel to kube-proxy and then back to the kernel so it results in higher throughput and better tail latency. The main downside is that it is more difficult to debug, because instead of a local binary that writes a log to /var/log/kube-proxy you have to inspect logs from the kernel processing iptables rules.
• IPVS proxy mode - IPVS is a Linux kernel feature that is specifically designed for load balancing. In IPVS mode, kube-proxy programs the IPVS load balancer instead of using iptables. This works, it also uses a mature kernel feature and IPVS is designed for load balancing lots of services; it has an optimized API and an optimized look-up routine rather than a list of sequential rules.

You can read more here - good question about proxy mode on StackOverflow, here - comparing proxy modes and here - good article about proxy modes.

---

Like rohatgisanat mentioned in his answer you can also use service mesh. Here is also good article about Kubernetes service mesh comparsion.