implement peer discovery logic through DNS in Golang using SRV records in Kubernetes cluster k8s

Question

I am trying to implement peer discovery logic through DNS in go using SRV records in cluster. I have headless service and statefulset pods ready and I am able to list all SRV records by using

kubectl run -it srvlookup --image=tutum/dnsutils --rm --restart=Never -- dig SRV demoapp.default.svc.cluster.local

but the following code does not work in cluster:

func pingdns() (url string) {
log.Println("start ping demoapp.default.svc.cluster.local.")
_, addrs, err := net.LookupSRV("dns-tcp", "tcp", "demoapp.default.svc.cluster.local")
if err != nil {
    log.Println(err.Error())
    return "dns wrong"
}
fmt.Println(addrs)
return "dns done."

}

error output:

lookup _dns-tcp._tcp.demoapp.default.svc.cluster.local on 10.96.0.10:53: no such host

I found example in this k8s-in-action book but it is written in NodeJS. How to do it in Golang ?

const dns = require('dns');
const dataFile = "/var/data/kubia.txt";
const serviceName = "kubia.default.svc.cluster.local";
const port = 8080;
...
var handler = function(request, response) {
  if (request.method == 'POST') {
...
} else {
    response.writeHead(200);
    if (request.url == '/data') {
      var data = fileExists(dataFile)
        ? fs.readFileSync(dataFile, 'utf8')
        : "No data posted yet";
      response.end(data);
    } else {
      response.write("You've hit " + os.hostname() + "\n");
      response.write("Data stored in the cluster:\n");
      dns.resolveSrv(serviceName, function (err, addresses) {
The app performs a DNS lookup to obtain SRV records.
   if (err) {
  response.end("Could not look up DNS SRV records: " + err);
  return;
}
var numResponses = 0;
if (addresses.length == 0) {
  response.end("No peers discovered.");
} else {
addresses.forEach(function (item) { ...

cong · Answer 1 · 2020-05-13T04:17:19.777

Thanks Shubham ! I read the medium post and found using grpc to make connection to SRV should browse all IPs as a RR fashion. But I am still looking for get all IPs.

Medium article: https://medium.com/google-cloud/loadbalancing-grpc-for-kubernetes-cluster-services-3ba9a8d8fc03

Gitrepo: https://github.com/jtattermusch/grpc-loadbalancing-kubernetes-examples#example-1-round-robin-loadbalancing-with-grpcs-built-in-loadbalancing-policy

import (
   "google.golang.org/grpc/balancer/roundrobin"
   "google.golang.org/grpc/credentials"
)

conn, err := grpc.Dial("dns:///be-srv-lb.default.svc.cluster.local", grpc.WithTransportCredentials(ce), grpc.WithBalancerName(roundrobin.Name))
c := echo.NewEchoServerClient(conn)

It makes calls on a list of IPs one by one. RR

Creating channel with target greeter-server.default.svc.cluster.local:8000
Greeting: Hello you (Backend IP: 10.0.2.95)
Greeting: Hello you (Backend IP: 10.0.0.74)
Greeting: Hello you (Backend IP: 10.0.1.51)

I have found my main issue is related to this problem.

Why golang Lookup*** function can't provide a server parameter?

I think stateless service implementation is to provide you pod ips, You can look for other golang resolver functions to get the list of ips from srv resolution. — Shubham Singh, May 14 '20 at 10:23

score 0 · Answer 2 · answered May 04 '20 at 22:21

It looks like you are looking for headless service.

Look for an example implementation: https://medium.com/google-cloud/loadbalancing-grpc-for-kubernetes-cluster-services-3ba9a8d8fc03

Kubernetes documentation: https://kubernetes.io/docs/concepts/services-networking/service/#headless-services for your reference.

To explain this: Kubernetes has a few implementations for services basically Endpoint based service and Headless service. Endpoint based services could be either with or without selectors. LoadBalancers are generally provided by Cloud providers.

Headless service are designed for client-side load balancing implementation. It looks like you are trying to implement your own DNS driven client-side load balancer.

ok, one design is to create one headless service per pod. so pod could communicate with each other using headless without specifying any LB algorithm — cong, May 13 '20 at 07:21

implement peer discovery logic through DNS in Golang using SRV records in Kubernetes cluster k8s

2 Answers2