k8s ExternalName endpoint not found - but working

Question

I deployed a simple test ingress and an externalName service using kustomize. The deployment works and I get the expected results, but when describing the test-ingress it shows the error: <error: endpoints "test-external-service" not found>. It seems like a k8s bug. It shows this error, but everything is working fine.

Here is my deployment:

kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: platform
resources:
  - test-ingress.yaml
  - test-service.yaml
generatorOptions:
  disableNameSuffixHash: true

test-service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: test-external-service
  namespace: platform
spec:
  type: ExternalName
  externalName: "some-working-external-elasticsearch-service"

test-ingress.yaml:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    kubernetes.io/ingress.class: nginx-external
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_cache_bypass $http_upgrade;
spec:
  rules:
    - host: testapi.mydomain.com
      http:
        paths:
          - path: /
            backend:
              serviceName: test-external-service
              servicePort: 9200

Here, I connected the external service to a working elasticsearch server. When browsing to testapi.mydomain.com ("mydomain" was replaced with our real domain of course), I'm getting the well known expected elasticsearch results:

{
  "name" : "73b40a031651",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "Xck-u_EFQ0uDHJ1MAho4mQ",
  "version" : {
    "number" : "7.10.1",
    "build_flavor" : "oss",
    "build_type" : "docker",
    "build_hash" : "1c34507e66d7db1211f66f3513706fdf548736aa",
    "build_date" : "2020-12-05T01:00:33.671820Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

So everything is working. But when describing the test-ingress, there is the following error:

test-external-service:9200 (<error: endpoints "test-external-service" not found>)

What is this error? Why am I getting it even though everything is working properly? What am I missing here?

After running `kubectl describe ing` command, do you have any warning/error entries in the `Events` section ? If you don't have any warning/error entries, everything should work as expected. — matt_j, Apr 20 '21 at 13:47

matt_j · Accepted Answer · 2021-04-20T17:26:57.343

This is how the kubectl describe ingress command works.
The kubectl describe ingress command calls the describeIngressV1beta1 function, which calls the describeBackendV1beta1 function to describe the backend.

As can be found in the source code, the describeBackendV1beta1 function looks up the endpoints associated with the backend services, if it doesn't find appropriate endpoints, it generate an error message (as in your example):

func (i *IngressDescriber) describeBackendV1beta1(ns string, backend *networkingv1beta1.IngressBackend) string {
    endpoints, err := i.client.CoreV1().Endpoints(ns).Get(context.TODO(), backend.ServiceName, metav1.GetOptions{})
    if err != nil {
        return fmt.Sprintf("<error: %v>", err)
    }
...

In the Integrating External Services documentation, you can find that ExternalName services do not have any defined endpoints:

ExternalName services do not have selectors, or any defined ports or endpoints, therefore, you can use an ExternalName service to direct traffic to an external service.

So basically you are saying, it works as expected. working, with an irrelevant error. Sounds like a bug as design :) — TomG, Apr 20 '21 at 16:15
You are right :) I updated my first sentence to be more appropriate to the situation. — matt_j, Apr 20 '21 at 17:27

Yiadh TLIJANI · Answer 2 · 2021-04-20T12:18:29.600

Service is a Kubernetes abstraction that uses labels to chose pods to route traffic to.

Endpoints track the IP Addresses of the objects the service send traffic to. When a service selector matches a pod label.

This is the case with Kubernetes service with the type ClusterIP, NodePort or LoadBalancer.

For your case, you use a Kubernetes service with the type ExternalName where the endpoint is a server outside of your cluster or in a different namespace, thus kubernetes displays that error message when you try to describe the ingress.

Usually we do not create an ingress that points to a service of type ExternalName because we are not supposed to expose externally a service that it is already exposed. The kubernetes ingress expects a service with the type ClusterIP, NodePort or LoadBalancer, that is why you got that unexpected error when you described the ingress.

If you are browsing that ExternalName within the cluster, it would be better to avoid using an ingress and use the service uri instead (test-external-service.<namespace>.svc.cluster.local:9200)

Anyway if you insist on using the Ingress, you can create a Headless service without selector and then manually create an endpoint using the same name as of the service. Follow the example here

As you assumed, it's a service outside the cluster, so I can't use a service URI. But if it doesn't expect an `ExternalName` type, don't let me use it. And if you do let me use it, why the error? Especially why the error if it does work as expected. Sorry, I don't understand your answer. — TomG, Apr 19 '21 at 19:46

k8s ExternalName endpoint not found - but working

2 Answers2