Exposing UDP port via Ingress-Nginx in GKE not working

Question

I am using the 1.25.8-gke.500 with ingress-nginx installed via helm chart 4.7.0 and seek to expose publicly SIP-UDP (5060) port which I do not manage to do. SIP-TCP (5060) on the other hand with the same setup below works. I am successfully using the same ingress-nginx for normal 80/443 HTTP exposure already.

ArgoCD Application:

  sources:
    - repoURL: "https://kubernetes.github.io/ingress-nginx"
      chart: ingress-nginx
      targetRevision: 4.7.0

Helm Value:

# -- UDP service key-value pairs
## Ref: https://github.com/kubernetes/ingress-nginx/blob/main/docs/user-guide/exposing-tcp-udp-services.md
udp:
  5060: "kamailio/lb-service:5060"

The port is correctly added into the ingess-nginx service

NAME                       TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)                                     AGE
ingress-nginx-controller   LoadBalancer   10.40.12.112   XX.XX.XX.XX   80:32278/TCP,443:30585/TCP,5060:31380/UDP   15d

The first issue I see is that in the auto created firewall rule the port 5060 is been added but as TCP instead of UDP.

k8s-fw-af50e8ac76ae44f13b3cf279afccfa67
Description{"kubernetes.io/service-name":"ingress-nginx/ingress-nginx-controller", "kubernetes.io/service-ip":"XX.XX.XX.XX"}
...
Protocols and ports
tcp:80
tcp:443
tcp:5060

Manually adding the port does not solve my issue. For completeness of the setup, the k8 service to the sip service below

NAME                  TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                      AGE
lb-service   ClusterIP   10.40.9.180   <none>        5060/UDP,5060/TCP,5061/TCP   13d

apiVersion: v1
kind: ConfigMap
  name: ingress-nginx-udp
  namespace: ingress-nginx
data:
  '5060': 'kamailio/lb-service:5060'

apiVersion: v1
kind: Service
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: LoadBalancer
metadata:
  annotations: null
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.8.0
    helm.sh/chart: ingress-nginx-4.7.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack
  loadBalancerIP: XX.XX.XXX.XXX
  ports:
    - appProtocol: http
      name: http
      port: 80
      protocol: TCP
      targetPort: http
    - appProtocol: https
      name: https
      port: 443
      protocol: TCP
      targetPort: https
    - name: 5060-udp
      port: 5060
      protocol: UDP
      targetPort: 5060-udp

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.8.0
    helm.sh/chart: ingress-nginx-4.7.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  minReadySeconds: 0
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.8.0
        helm.sh/chart: ingress-nginx-4.7.0
    spec:
      containers:
        - args:
            - /nginx-ingress-controller
            - '--publish-service=$(POD_NAMESPACE)/ingress-nginx-controller'
            - '--election-id=ingress-nginx-leader'
            - '--controller-class=k8s.io/ingress-nginx'
            - '--ingress-class=nginx'
            - '--configmap=$(POD_NAMESPACE)/ingress-nginx-controller'
            - '--udp-services-configmap=$(POD_NAMESPACE)/ingress-nginx-udp'
            - '--validating-webhook=:8443'
            - '--validating-webhook-certificate=/usr/local/certificates/cert'
            - '--validating-webhook-key=/usr/local/certificates/key'
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
          image: >-
            registry.k8s.io/ingress-nginx/controller:v1.8.0@sha256:744ae2afd433a395eeb13dc03d3313facba92e96ad71d9feaafc85925493fee3
          imagePullPolicy: IfNotPresent
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          livenessProbe:
            failureThreshold: 5
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          name: controller
          ports:
            - containerPort: 80
              name: http
              protocol: TCP
            - containerPort: 443
              name: https
              protocol: TCP
            - containerPort: 8443
              name: webhook
              protocol: TCP
            - containerPort: 5060
              name: 5060-udp
              protocol: UDP

Thankful for any pointers or directions where I am going wrong...

I read https://stackoverflow.com/questions/75036656/how-do-i-expose-a-udp-service-externally-on-google-kubernetes-engine-gke-using but it has no solution provided but the explained issue is the same in my case... — Stefan Walther, Jun 22 '23 at 07:22
Please let me know whether the shared info was helpful. I am happy to assist if you have any further queries. If the provided information was helpful,you can check this link [What should I do when someone answers my question](https://stackoverflow.com/help/someone-answers)? — Fariya Rahmat, Jun 27 '23 at 03:34
Did you get the time to check my answer? I hope the shared information was helpful to you in overcoming the issue. — Fariya Rahmat, Jul 27 '23 at 06:58

Fariya Rahmat · Answer 1 · 2023-06-23T10:23:36.860

0

As mentioned in the document written by Faisal Memon we can understand that since 1.9.13 NGINX provides UDP Load Balancing.

To expose UDP service via NGINX, you need four things:

Add port definition to DaemonSet (by default it only exposes TCP/80 and TCP/443)
Run your app
Create a service exposing your app
Add service definition to ConfigMap udp-services in the ingress-nginx namespace.

Try running:

apiVersion: v1
kind: ConfigMap
metadata:
  name: ingress-nginx-udp
  namespace: ingress-nginx
data:
  5060: "kamailio/lb-service:5060"

Refer to the link for more information.

edited Jun 23 '23 at 10:23

answered Jun 22 '23 at 10:02

Fariya Rahmat

2,123
3
11

Refer to the SO [link](https://stackoverflow.com/questions/61430311) which may help you to resolve the issue. – Fariya Rahmat Jun 22 '23 at 10:15
I have extended my question, the reference document is to my understanding all correctly in place (generated by the Helm Chart). The target port in the Loadbalancer reads odd to me but it is also auto generated: - name: 5060-udp port: 5060 protocol: UDP targetPort: 5060-udp – Stefan Walther Jun 22 '23 at 11:05
As per my observation you have not added meta data in your configuration and in data filed you have used double( " )quotes also there is no need to use( ' ) single quotes in agrs filed. Similarly review your configuration thoroughly. I have updated my answer .Let me know if that helps. – Fariya Rahmat Jun 23 '23 at 10:27
unfo. does this not resolve the issue or I have a massive understanding problem. I see that it binds the node ports but on the public IP assigned to the ingress controller not traffic is received. ' or " are mutuals exclusive and in my case render though the helm chart... – Stefan Walther Jul 28 '23 at 09:29

Exposing UDP port via Ingress-Nginx in GKE not working

1 Answers1