I feel like I am close but I am not sure if I am just terminating wrong with the ingress controller and what I am trying is not possible. Or, the certificate is in the way and I need to do something to alleviate the issue.
Another idea I see is using a HAPROXY LB in front of the ingress-nginx controller.
First, I am using the ingress Nginx tcp/udp controller for Kubernetes described here with an Letsencrypt RA CA
https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
This allows me through my HELM installation to create a tcp config and mapping.
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx --create-namespace --namespace $NAMESPACE --set controller.service.annotations.service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path=/healthz,controller.service.annotations.service\.beta\.kubernetes\.io/azure-dns-label-name=$DNS_LABEL --set controller.service.loadBalancerIP=$STATIC_IP --set tcp.18083=$NAMESPACE/emqx-ee:18083,tcp.8883=$NAMESPACE/emqx-ee:8883
That does allow for an mqtts:// ssl connection to go through all the way to the 8883 backend pod. The issue is when I want to terminate at the loadbalancer and send the resulting incoming traffic to terminate the TLS and go to the TCP 1883 port.
To try this I change the port tcp definition
from this:
'8883': ingress-emqx/emqx-ee:8883
to this:
'8883': ingress-emqx/emqx-ee:1883
When I connect the client mqttx to 8883 CA I get the resulting error:
Error: Client network socket disconnected before secure TLS connection was established
What exactly does the termination for ingress-nginx? Is it only the ingress instruction rule or is it the controller TCP Proxy Protocol?
https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/ https://kubernetes.github.io/ingress-nginx/examples/tls-termination/
Is the only way to terminate is to utilize the ingress instruction/rewrite rule? If so, I think this is where the controller won't work because the route ingress rule only works with http and won't act upon mqtt layer-7 protocol.
Is there a way for the ingress to align with the ingress-controller service? I'm not sure it even matters to the previous issue. I simply want to take the incoming 8883 port and switch it to the resulting 1883 port and IP.
