17

Can anybody let me know how can we access the service deployed on one pod via another pod in a kubernetes cluster?

Example:

There is a nginx service which is deployed on Node1 (having pod name as nginx-12345) and another service which is deployed on Node2 (having pod name as service-23456). Now if 'service' wants to communicate with 'nginx' for some reason, then how can we access 'nginx' inside the 'service-23456' pod?

suren
  • 7,817
  • 1
  • 30
  • 51
Aditya Datta
  • 567
  • 2
  • 7
  • 17
  • 1
    you are not explaining yourself properly. What is a 'service' for you? Kubernetes has flat networking by default, so all pods and nodes can talk to each other, no matter their namespaces. – suren Nov 21 '18 at 11:22
  • I meant about any random service. Services are just a mechanism of accessing the deployments . The comments in the below section clearly describe the issue. What I want to know is that if any service such as nginx is deployed on a pod (say pod 1) and another service named eureka is deployed on second pod (say pod 2), then how can we access nginx from pod 2 ? I am able to access services in the master server but I am not able to access in the corresponding pods. – Aditya Datta Nov 21 '18 at 12:29
  • OK. So, as I said, k8s networking is flat, so you should be able to talk from one pod to another. How did you create the cluster? If you followed any doc, can you paste is here? – suren Nov 21 '18 at 14:15
  • Hi Suren , I followed this link ---- https://www.howtoforge.com/tutorial/centos-kubernetes-docker-cluster/ – Aditya Datta Nov 22 '18 at 07:23

4 Answers4

10

There are various ways to access the service in kubernetes, you can expose your services through NodePort or LoadBalancer and access it outside the cluster.

See the official documentation of how to access the services.

Kubernetes official document states that:

Some clusters may allow you to ssh to a node in the cluster. From there you may be able to access cluster services. This is a non-standard method, and will work on some clusters but not others. Browsers and other tools may or may not be installed. Cluster DNS may not work.

So access a service directly from other node is dependent on which type of Kubernetes cluster you're using.

EDIT:

Once the service is deployed in your cluster you should be able to contact the service using its name, and Kube-DNS will answer with the correct ClusterIP to speak to your final pods. ClusterIPs are governed by IPTables rules created by kube-proxy on Workers that NAT your request to the final container’s IP.

The Kube-DNS naming convention is service.namespace.svc.cluster-domain.tld and the default cluster domain is cluster.local.

For example, if you want to contact a service called mysql in the db namespace from any namespace, you can simply speak to mysql.db.svc.cluster.local.

If this is not working then there might be some issue with kube-dns in your cluster. Hope this helps.

EDIT2 : There are some known issue with dns resolution in ubuntu, Kubernetes official document states that

Some Linux distributions (e.g. Ubuntu), use a local DNS resolver by default (systemd-resolved). Systemd-resolved moves and replaces /etc/resolv.conf with a stub file that can cause a fatal forwarding loop when resolving names in upstream servers. This can be fixed manually by using kubelet’s --resolv-conf flag to point to the correct resolv.conf (With systemd-resolved, this is /run/systemd/resolve/resolv.conf). kubeadm 1.11 automatically detects systemd-resolved, and adjusts the kubelet flags accordingly.

Prafull Ladha
  • 12,341
  • 2
  • 37
  • 58
  • Hi Praful , Sorry if my question was not clear. I have reframed the question. I actually want to access the service which is hosted in one pod via another pod . Is there any way we can do it ? I am using flannel network here . – Aditya Datta Nov 21 '18 at 06:47
  • Hi Praful , You are right regarding the DNS convention. I have deployed 'eureka' on one of the pods and from the master server, I am able to do nslookup and query DNS and it returns the result as the cluster IP for 'eureka-server.default.svc.cluster.local'. I fired command as 'nslookup eureka-server.default.svc.cluster.local 10.96.0.10' but when I am firing the same command from any of the pods, it returns as 'could not resolve host; although the name server for the pods is also set as 10.96.0.10 . – Aditya Datta Nov 21 '18 at 10:37
  • Hi Aditya, Following is the guide to debug the dns resolution, could you please check if everything is working fine on your end: https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/ – Prafull Ladha Nov 21 '18 at 10:42
  • Hi Praful , I checked the article and found out everything is running. I am able to see DNS logs but only if I run the DNS query from the master server. When I am doing nslookup from the corresponding pods, then it is not able to connect and correspondingly no DNS logs are generated also. I think the pods are not referring '/etc/resolv.conf' file . – Aditya Datta Nov 22 '18 at 07:22
  • My master server is CentOS7 and both of my pods are Ubuntu systems. I think there might be an issue with Ubuntu . – Aditya Datta Nov 22 '18 at 07:28
  • Hi Aditya, There is an known issue with ubuntu distribution in dns resolution, I have edited my answer. Please have a look, – Prafull Ladha Nov 22 '18 at 10:48
  • Hi Praful , one question. I saw that DNS service is accessible from the master server but not accessible from the pod. I just now checked that the DNS service is also not accessible from the slave node on which this pod is hosted. Can you tell me if this behaviour is correct or DNS should be accessible from the slave node also ? – Aditya Datta Nov 22 '18 at 11:20
  • Ideally, In K8S every pod should be able to access another pod, hence the dns service should be accessible from all the pods. This dns might not work from worker node(some cluster provides it some not) which is completely fine, if it doesn't work from node. So if you're not able to access dns service from pods, your cluster has some issues. Could you please specify the k8s version you're using. – Prafull Ladha Nov 22 '18 at 11:37
  • Its kubernetes version 1.11.0 . – Aditya Datta Nov 22 '18 at 11:41
  • I will follow guide you used and try to setup the cluster when I get time. I will use centos(only vm available) and update the answer. – Prafull Ladha Nov 22 '18 at 11:43
6

Did you expose your deployment as a service? If so, simply access it by it's dns name, like http://nginx-1234 - or if it's in a different namespace: http://nginx-1234.default.svc (change "default" to the namespace the service lives in) or http://nginx-1234.default.svc.cluster.local

Now if you did NOT expose a service, then you probably should. You don't need to expose it to the outside world, simply don't define a service type and it will only be available inside your cluster.

If for some reason you don't want to expose a service (can't think of any reason), you can query the api server for the pod IP. You will need to provide a token for authentication, but these are available inside the pod:

get the token:

TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)

call the api server:

curl https://kubernetes.default.svc/api/v1/namespaces/default/pods--silent \
     --header "Authorization: Bearer $TOKEN" --insecure

you can refine your query by adding ?fieldSelector=spec.nodeName%3Dtargetnodename or similar (simply use a json path). the output can be parsed with https://stedolan.github.io/jq/ or any other JSON utility.

Markus Dresch
  • 5,290
  • 3
  • 20
  • 40
  • Hi Markus, I exposed my deployment as a service and I am using NodePort for this. I hae deployed eureka on one of the nodes and when I do nslookup from the master server using '10.96.0.10' as the name server, it returns the correct result as the full FQDN name . But when I fire the nslookup command from any of the pods, it shows as 'Could not resolve host' although the '/etc/resolve.conf' file of the pod shows '10.96.0.10' as the name server . – Aditya Datta Nov 21 '18 at 10:40
  • i don't know about eureka, but normally you don't have to change /etc/resolve.conf for communicating between services in your cluster. – Markus Dresch Nov 21 '18 at 11:34
1

I faced a similar issue, the following link might solve your issue. Generally, all of the services are visible and accessible within the cluster. Expose your service-23456 service to type ClusterIP and to port 8080. Then you can call endpoint 'http://service-23456:8080' from nginx service.

Unable to communicate between 2 node,js apps in Istio enabled GKE cluster

Niel Godfrey Pablo Ponciano
  • 9,822
  • 1
  • 17
  • 30
Roobal Jindal
  • 214
  • 2
  • 13
0

Simlar question was answered here: Kubernetes - How to acces to service from a web server in pod with a rest request

Just replace "ProductWebApp" with "nginx" and "DashboardWebApp" with "service".

apisim
  • 4,036
  • 1
  • 10
  • 16