26

Kubernetes admin can use --cluster-domain to customize cluster domain instead of using default one: cluster.local Kubelet Configs.

So the question is, how does an application pod check this domain in runtime?

Rico
  • 58,485
  • 12
  • 111
  • 141
Andy Luo
  • 305
  • 1
  • 4
  • 8

3 Answers3

19

It needs to be configured on the DNS server.

Either kube-dns or coredns (Favored on newer K8s versions)

kube-dns: it's a cli option --domain

core-dns: you can configure the K8s ConfigMap

And you see here:

The kubelet passes DNS to each container with the --cluster-dns= flag.

If you'd like to know how a pod resolves cluster.local it does it through the /etc/resolv.conf that the kubelet mounts on every pod. The content is something like this:

$ cat /etc/resolv.conf
nameserver 10.96.0.10
search <namespace>.svc.cluster.local svc.cluster.local cluster.local <nod-domain>
options ndots:5

10.96.0.10 is your coredns or kube-dns cluster IP address.

Rico
  • 58,485
  • 12
  • 111
  • 141
  • 2
    Thanks Rico. That's not what I want though. Basically, I know that it's kubelet that configures the domain with the flag --cluster-domain=. And that is what I want to query from within a running pod. – Andy Luo Oct 30 '18 at 03:28
  • What exactly are you looking for? How does a pod know to search for `cluster.local`? Added more details let me know if that's what you need. – Rico Oct 30 '18 at 04:20
  • Yes, the replacement for the default ? I see from your added answer, you are implying parsing /etc/resolv.conf? is the format in /etc/resolv.conf always fixed? I can't find any spec covering that area. – Andy Luo Oct 30 '18 at 04:32
  • Yeah, it's always there. The spec is just the linux man page :) http://man7.org/linux/man-pages/man5/resolv.conf.5.html. This gets by the kubelet on every container and the values are populated depending on the configs/namespace. Unfortunately running `hostname` on the pod will only give you the pod name and the pod-name.domain. There's no k8s pod spec field that says this is the domain for this pod – Rico Oct 30 '18 at 14:47
  • Thanks Rico. It feels a bit hacky, but as long as it works. Though it would really be handy to be able to just query it like query namespace. – Andy Luo Oct 30 '18 at 23:56
  • Yes. You can file an issue with on the Kubernetes github repo – Rico Dec 05 '19 at 05:16
  • @Rico, a related question. The nameserver in my pods' resolve.conf is not my coredns cluster IP. So it gives errors. How can I fix that? Basically where is "--cluster-dns" flag set? – prime Nov 26 '21 at 20:40
9

Running a DNS query against service kubernetes.default is a possible solution. Here is a one-liner example in shell:

kubectl run -it --image=ubuntu --restart=Never shell -- \
sh -c 'apt-get update > /dev/null && apt-get install -y dnsutils > /dev/null && \
nslookup kubernetes.default | grep Name | sed "s/Name:\skubernetes.default//"'

This will returns as last line:

.svc.cluster.local

However, I think it would be more robust to implement this algorithm in a programming language like go which have a good DNS client implemented in net library, here is an example you can run in a pod:

package main

import (
    "fmt"
    "net"
    "strings"
)

// GetClusterDomain returns Kubernetes cluster domain, default to "cluster.local"
func getClusterDomain() string {
    apiSvc := "kubernetes.default.svc"

    cname, err := net.LookupCNAME(apiSvc)
    if err != nil {
        defaultClusterDomain := "cluster.local"
        return defaultClusterDomain
    }

    clusterDomain = strings.TrimPrefix(cname, apiSvc)
    clusterDomain = strings.TrimSuffix(clusterDomain, ".")

    return clusterDomain
}

func main() {
    fmt.Println(getClusterDomain())
}
Fabrice Jammes
  • 2,275
  • 1
  • 26
  • 39
  • It will not work if you have 'ndots:1' at least. Preferably would be great to have an result of API call. – tamerlaha Jul 16 '20 at 22:38
  • 1
    In Kubernets 'ndots' is set to 5 by default so it works fine. If 'ndots' is set to 1 Kubernetes will not be able to perform its DNS resolution for local names at all, so 'ndots:1' would break Kubernetes DNS. An API call would be convenient but it would add some dependencies between application pods and k8s and this is not recommended as a best practice. So I do not see an easy solution here and I live with technique above on our large scale production platforms: https://confluence.lsstcorp.org/display/DM/Understanding+and+managing+k8-based+deployment+of+Qserv+at+NCSA – Fabrice Jammes Jul 17 '20 at 07:57
1

For the clusters which has coredns, I have a command for querying the cluster domain name over CoreDNS config map inside kube-system namespace.

kubectl get cm coredns -n kube-system -o jsonpath="{.data.Corefile}" \
  | grep ".local " \
  | awk -F ' ' '{print $2}'
tuna
  • 136
  • 7