Kubernetes nginx ingress can't access pod in different namespace

Question

I am trying to setup the kuard demo app in the namespace example-ns exposed by nginx ingress.
Exposing it in the default namespace works but when I expose it in the namespace example-ns I get:
```503 Service Temporarily Unavailable```

These are to service, deployment and ingress yamls I use for kuard:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kuard
  namespace: example-ns
spec:
  selector:
    matchLabels:
      app: kuard
  replicas: 1
  template:
    metadata:
      labels:
        app: kuard
    spec:
      containers:
      - image: gcr.io/kuar-demo/kuard-amd64:1
        imagePullPolicy: Always
        name: kuard
        ports:
        - containerPort: 8080

---

apiVersion: v1
kind: Service
metadata:
  name: kuard
  namespace: example-ns
spec:
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
  selector:
    app: kuard

---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kuard
  namespace: example-ns
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: htpasswd
    nginx.ingress.kubernetes.io/auth-realm: "Enter your credentials"

spec:
  tls:
  - hosts:
    - example.mydomain.dev
    secretName: quickstart-example-tls
  rules:
  - host: example.mydomain.dev
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kuard
            port:
              number: 80

As you can see everything is in the same namespace and describing the ingress results in:

❯ kubectl describe ingress kuard -n example-ns
Name:             kuard
Labels:           <none>
Namespace:        example-ns
Address:          192.168.69.1
Ingress Class:    <none>
Default backend:  <default>
TLS:
  quickstart-example-tls terminates example.mydomain.dev
Rules:
  Host                    Path  Backends
  ----                    ----  --------
  example.mydomain.dev
                          /   kuard:80 (10.69.58.226:8080)
Annotations:              cert-manager.io/cluster-issuer: letsencrypt-prod
                          kubernetes.io/ingress.class: nginx
                          nginx.ingress.kubernetes.io/auth-realm: Enter your credentials
                          nginx.ingress.kubernetes.io/auth-secret: htpasswd
                          nginx.ingress.kubernetes.io/auth-type: basic
Events:
  Type    Reason             Age                From                       Message
  ----    ------             ----               ----                       -------
  Normal  CreateCertificate  28m                cert-manager-ingress-shim  Successfully created Certificate "quickstart-example-tls"
  Normal  Sync               27m (x2 over 28m)  nginx-ingress-controller   Scheduled for sync
  Normal  Sync               27m (x2 over 28m)  nginx-ingress-controller   Scheduled for sync

I also read same issues like this but this solution is not working as seen here.
Anyone has an idea whats wrong here?
Thanks in advance!

SOLUTION:

I checked the logs of the ingress controller and saw that the auth secret was in the default namespace. Thats why only pods from default namespace were acessible. Moving the secret into the proper namespace solved the issue!

score 1 · Accepted Answer · answered Oct 24 '22 at 09:12

First of all you should not use the Annotation kubernetes.io/ingress.class anymore as it's deprecated. Instead use .spec.ingressClassName to refer to your desired Ingress Controller:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-myservicea
spec:
  ingressClassName: nginx
  rules:
  ...

It seems like that your Ingress in your desired Namespace can't seem to sync with the Controller, so if there is any Netpols in your example-ns and the Namespace where your Controller resides; back them up and delete them, to make sure the connection isn't being blocked.

Next you should check the logs of your Ingress Controller itself, if the connection reaches it; you will see surely the reason in the logs why the Ingress resource doesn't work. Also sharing your Config for the Ingress Controller would be helpful.

Kubernetes nginx ingress can't access pod in different namespace

1 Answers1