1

We have a multi tenant application and for each tenant we provision separate container image. Likewise we create a subdomain for each tenant which will be redirected to its own container. There might be a scenario where 1000s of tenants can exist and its dynamic.

So it has become necessary for us to consider the limitations in ingress controllers for Kubernetes in general before choosing. Especially the nginx-ingress.

  1. Is there any max limitation on number of Ingress resources or rules inside ingress that can be created? Or will there be any performance or scaling issues when too many ingress resources are created ?

  2. Is it better to add a new rule(for each subdomain) in same ingress resource or to create separate ingress resource for each subdomain ?

Arun Ramachandran
  • 510
  • 5
  • 14

1 Answers1

2

AFAIK, there are no limits like that, You will either run out of resources or find a choke point first. This article compares resource consumption of several loadbalancers.

As for Nginx-ingress there are few features hidden behind paid nginx plus version as listed here.

If You wish to have dynamic configurations and scalability, You should try envoy based ingress like Ambassador or Istio.

Envoy offers dynamic configuration updates which will not interrupt existing connections. More info here.

Check out this article which compares most of popular kubernetes ingress controllers.

This article shows a great example of pushing HAproxy and Nginx combination to its limits.

Hope it helps.

Piotr Malec
  • 3,429
  • 11
  • 16
  • thank you @piotr. very useful links. It will help us in taking informed decision. Apart from that I got to know from community that etcd has request limit of 1.5mb by default. If the ingress resource exceeds that limit it might come as an limitation. – Arun Ramachandran Jul 06 '20 at 18:03
  • Yes that is true, note that the size limit is for the config map that is generated from ingress object manifest. Which is key - value pair and which in 1.5mb of You could fit a lot of ingress rules. More info [here](https://stackoverflow.com/questions/53012798/kubernetes-configmap-size-limitation). – Piotr Malec Jul 07 '20 at 11:14