2

I've been experimenting with an ICP instance (ICP 2.1.0.2): 1 master node and 2 worker nodes.

I noticed that the pods in my ICP Kubernetes cluster don't have outbound Internet connectivity (or are having DNS lookup issues)

For example, If I start up a busybox pod in my cluster, and try to do "nslookup github.com" or "ping google.com" .. it fails..

kubectl run curl --image=radial/busyboxplus:curl -i --tty

root@curl-545bbf5f9c-gssbg:/ ]$ nslookup github.com
Server:    10.0.0.10
Address 1: 10.0.0.10

nslookup: can't resolve 'github.com'

I checked and saw that "kube-dns" (service, pod, daemonset.extensions, daemonset.apps) does appear to be running.

When I'm logged into (eg. SSH) to the ICP master and the worker nodes machines, I am able to ping these external sites successfully.

Any suggestions for how to troubleshoot this problem? Thanks!

Dtma
  • 21
  • 3

1 Answers1

1

We had kind of the reverse problem - where we could look up anything on internet or other domains, but not the domain in which the cluster was deployed.

That turned out to be the vague documentation around what cluster_domain and cluster_CA_domain mean in the config.yaml. But as a plus we got to learn a bit more about those and about configuring kube-dns.

Basically cluster_domain should be a private virtual domain to the cluster for which kube-dns will be authoritative. Anything else it should use the host's resolve.conf nameservers as upstream servers. If you suspect that your DNS servers are not being utilised for public DNS then you can update the kube-dns configMap to specify the upstream servers that it should use.

https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/

This is assuming you have configure cluster_domain, cluster_CA_domain correctly of course.

They should look something like

cluster_domain = mycluster.icp  <----- could be "Mickey-mouse" for all it matters
cluster_CA_domain = icp.mycompany.com <----- the endpoint that portal/registry/api etc are accessible to users on
Seer
  • 524
  • 1
  • 8
  • 19