AKS Ingress endpoint on Azure Traffic Manager

Question

I have deployed multiple microservices on an AKS cluster and exposed it on nginx ingress controller. The ingress pointing to a static ip with dns as blabla.eastus.azure.com

Application is exposed on blabla.eastus.azure.com/application/ and blabla.eastus.azure.com/application2/ .. etc.

I have created a Traffic manager profile in blabla.trafficmanager.net in Azure. How should i configure the AKS ingress in traffic manager such that traffic manager reroutes the request to an application deployed on AKS ingress.

---Ingress.yaml configuration used
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress
  namespace: ns
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - host: blabla.eastus.azure.com
    http:
      paths:
      - backend:
          serviceName: application1
          servicePort: 80
        path: /application1(/|$)(.*)
      - backend:
          serviceName: application2
          servicePort: 80
        path: /application2(/|$)(.*)
      - backend:
          serviceName: aks-helloworld
          servicePort: 80
        path: /(.*)

When i hit curl http://blabla.trafficmanager.net the response is default backend - 404

When i update the host to http://blabla.trafficmanager.net, i am able to access the application through http://blabla.trafficmanager.net\application1

The same is true for any custom cname created. I created a cname as custom.domain.com and redirected it to blabla.eastus.azure.com. So unless i update the host in ingress directly to custom.domain.com I am not able to access it through the custom domain

do you really need a traffic manager? ingress can do path based routing. — P Ekambaram, Dec 13 '19 at 09:51

score 2 · Answer 1 · answered Dec 20 '19 at 09:24

2

The actual request will never pass via Traffic Manager. Traffic Manager is a DNS based load balancing solution that is offered by Azure.

When you browse Azure TM endpoint, it resolves and gives you an IP. Then your browser request that IP address.

In your case, your AKS should have a Public Endpoint to which TM can resolve the DNS query. Also you need to create an CNAME record to map TM FQDN to your Custom Domain. If this is not done, you will get 404.

The above mentioned custom header settings are for the probes, but the actual request will be sent from the client browser to the endpoint/IP which the TM resolves to.

answered Dec 20 '19 at 09:24

msrini-MSIT

1,444
1
7
11

I have two AKS clusters in two regions having ngnix ingress LB with custom domain aks1.xyz.com and aks2.xyz.com respectivey. I have two services A and B on each cluster that is individually reachable at relative path /srv-a and /srv-b on each cluster via ingress host aks1.xyz.com and aks2.xyz.com. I added these custom domain in the Azure Traffic Manager with config to probe on port 443 and path /healthz. Endpoints are online, but i can not reach to the services via TM custom domain xyz.com mapped to xyz.trafficmanager.net. Please suggest!! – Rahul Mohan May 31 '20 at 08:11
Facing the same issue. Was there a solution for this? – ctienshi Jul 11 '23 at 10:24

score 1 · Answer 2 · answered Jul 21 '20 at 10:05

One approach to achieve your need is to rigidly control the traffic between public DNS and the Ingress Controller Public IP in each region; delegate the flexibility of how you publish services to the HTTP SNI protocol:

To keep it simple, the Ingress Controller does not have any A DNS record assigned to its public IP.

So, we'll implement the architecture from right to left following the diagram.

The traffic manager will have two endpoints: one per region. The value of each endpoint will be the corresponded Ingress public IP.

The DNS service will have configured a CNAME (alias) for app.mydomain.com as mine-apps.trafficmanager.net.

In this way, the client connecting to app.mydomain.com will resolve the Traffic Manager (TM) service, which is a Geo DNS, and based on the client's IP, will return to the client the closer target region between A and B.

In the same way, you can use the URL or path-based routing for exposing services via the Ingres and control how clients connect to them. Just make sure that your DNS is aware of how to connect to the Traffic Manager. The rest will be handled magically by TM and the Ingress object in Kubernetes.

Last but not least, once all the integrations are properly configured and they satisfy your primary need you can start to extend the existing architecture and adapt to your real requirements; for example: getting rid of static IPs in the Traffic Manager's endpoints.

Hi Andov! I'm curious to know more about this setup. So when the user/client connects to app.domain.com it will resolve to the TM URL and this is clear. And then TM will return the client with closer target (Either A or B) based on the geo-location. That part is also very clear. So eventually TM will return the Public IP address of NGINX Ingress Controller to the client. So then user connects to this public IP right? And then when the user access that public IP it will load 404 or something right? Can you explain that part a bit? — Container-Man, Jul 15 '23 at 07:35
Hi @Container-Man, correct the user will connect transparently through his client to the regional public IP of the Ingress Controller load balancer. It is desirable that the user hits the Pod hosted in the Kubernetes through the Ingress Controller. If 404 pops out to the end-user it means: HTTP Host header mismatch in Ingress Controller, a misconfiguration between the TM and the load balancer of the Ingress Controller, last but not least the Ingress object that points to the Kubernetes pod is missing. — andov, Jul 21 '23 at 13:52

AKS Ingress endpoint on Azure Traffic Manager

2 Answers2