3

So I am using the kubernetes NGINX Ingress Controller and installed it via the helm chart.

I didn't find any real documentation for the chart but referencing the values.yaml, I changed the controller.kind value to DaemonSet.

So the first part of this question, is this a good practice. I couldn't find any information about this, but one controller might not be enough to handle all incomming traffic, also we ran into some issues, where the node running ingress controller was down. I hope having the controller run as DaemonSet is sufficient, both regarding the availability and also regarding performance?

The second and more important part of the question is, how could I have multiple loadbalancers in such a setup? Since one loadbalancer might not be able to handle all traffic, we want to add multiple Loadbalancers, but as far as I understand, there will be one Service of type Loadbalancer created. How can I have multiple Loadbalancers?

Also is it possible to do this through a helm chart configuration?

iLuvLogix
  • 5,920
  • 3
  • 26
  • 43
natschz
  • 1,007
  • 10
  • 23
  • you might want to implement a supervisor-process overlooking multiple instances of loadbalancers which will redistribute traffic according to load and availability as well as respawning single instances once they are down/gone or spawning additional ones if needed.. – iLuvLogix Jan 18 '21 at 14:07
  • But wouldn't this supervisor process then become the new bottle neck? What i was thinking is havin multiple loadbalancers and then just distribute traffic via DNS load balancing and in the first step, add/remove ladbalancers manually. But I'm not sure how I could configure the helm chart so that there are multiple loadbalancers and how this would play out with the controllers and possible bottlenecks. – natschz Jan 18 '21 at 14:13
  • DNS load balancing is the suggested solution for a similar question: https://stackoverflow.com/questions/55201050/what-can-we-do-when-load-balancer-becomes-the-bottleneck – Krishna Chaurasia Jan 18 '21 at 14:19
  • I think this might help - https://stackoverflow.com/questions/48669961/can-kubernetes-ingress-nginx-be-autoscaled – CloudBalancing Jan 18 '21 at 14:21
  • @KrishnaChaurasia thank you - the question is how do get the nginx controller configured so i can use DNS load balancing – natschz Jan 18 '21 at 14:27
  • @CloudBalancing As far as I understand, that is what I already did, I deployed the controller with a DaemonSet (hope this will be sufficient). But the issue is that even tough I now have multiple nginx controllers, there still is only one Loadbalancer. I'm not sure how I can add multiple loadbalancers (trough the helm chart) and how this will play out, regarding the controllers (resources and routing). Do i have to just duplicatet the loadbalancer service, is this engough and how do I have to configure it. Tbh Really want to do it trough the helm chart (if possible). – natschz Jan 18 '21 at 14:34

1 Answers1

3

Nginx Ingress Controller is not a load balancer, so when you say one load balancer would not be enough, I guess you mean one Ingress Controller pod would not be enough. In that case, creating several load balancers would not really help, as you would get more cloud provider load balancers pointing to the same Ingress Controller.

Now, Having a DaemonSet for IC might seem a good idea, but it is not. Especially when you have tens or hundreds of nodes, in which case you might have tens or hundreds of IC pods that you don't really need. You might need 5, for 40 nodes.

The proper way would be to change the object back to Deployment, create a HorizontalPodAutoscaler object for that Deployment, to scale based on a threshold that you will set. You might need to install metrics server for that. If you have the issue of having a node autoscaler, for example, that might take a node down at any moment, you should set a PodDisruptionBudget object, to prevent these cases. PDBs will prevent a node from being drained, and the node autoscaler would take down another node. You could also set an Anti-affinity against itself to prevent two IC pods to be deployed on the same node, to do the extra mile.

suren
  • 7,817
  • 1
  • 30
  • 51
  • That sounds really promissing, so is it possible trough the helm chart i mentioned above, to create multiple loadbalancers? If not do i have to just duplicate the current loadbalancer? – natschz Jan 18 '21 at 15:11
  • @natschz you only need multiple load balancers if you need multiple IP addresses. What you need is multiple replicas of your Ingress Controller, and for that you only need the HPA and the PDB. No need to edit your helm chart. – suren Jan 18 '21 at 15:40
  • I think we do actually need multiple load balancers, I posted another [question](https://stackoverflow.com/questions/65775305/how-to-scale-loadbalancers-are-loadbalancers-a-bottleneck) today. I'm not sure how this gues with other cloud providers, but DigitalOcean only e.g. supports up to 40,000 simultaneous connections. To overcome this, as far as I understand, we need multiple loadbalancers? – natschz Jan 18 '21 at 15:45
  • @natschz in that case, yes you would get another one and do DNS balancing. – suren Jan 18 '21 at 18:43