8

I am searching for a solution with static private IPs for my container instances. I will add the Application Gateway to it to also have a static public IP for it.

I am checking https://stackoverflow.com/a/59168070/7267638 and it looks good until "Add the private IP of the container instance into the backend pool of the application gateway". The point which is not clear to me is what to do when I restart the container and add others in the meantime - it can end up with different private IPs.

I need to have them static not only to be able to configure backend pool for the Gateway, but also for internal routing purposes. Without some kind of static config, I would need to reconfigure all services after private IP change to be able to find each other again.

Maybe can I use some kind of internal DNS or use container names or so?

O1da
  • 143
  • 1
  • 10

3 Answers3

7

Private static IPs for ACI is (as of today) not supported. I don't think there is a real workaround here except for checking after a container has been (re-)started if the IP has been changed.

Your best bet might be to use subnets of the minimum required size when putting ACI into a subnet - and only use one ACI per subnet. This way the chance might be lower that the IP actually changes, but still no guarantees there.

silent
  • 14,494
  • 4
  • 46
  • 86
3

I have been having the same issue and solved it with the alternative @silent mentions. I created a 29'er subnet, which is the smallest you can create on Azure with 3 available addresses (the other 5 are reserved), per Azure Container Instance I am hosting. I register all three available addresses in the backend pool in application gateway, so that it can forward requests to the IP address of the instance. The built-in probing seems to just do this.

Bernd
  • 539
  • 3
  • 10
  • I did the same, but there are many issues with that. You can endup with IPs rotated if you are doing a deployment of 3 containers together. But most importantly we are having issues with the underlying Azure infra - from time to time 1 IP is leaking from the subnet and it's causing containers are pending we had to ask Azure support to do some kind of manual mitigation every time. So to sum it up, it's not a viable solution, we have it in prod but it's not reliable - Microsoft is strongly suggesting to onboard Kubernetes instead of this fragile solution :( – O1da Feb 23 '21 at 08:36
  • Thanks a million for your comment. I do not quite understand your issue with the three containers - is that in three different ACI instances or in the same? I have one container per instance per subnet per backend pool. So far, this behaves nicely. Just out of curiosity, who in Microsoft recommended you to use Kubernetes instead of ACI and what background? – Bernd Feb 23 '21 at 12:50
  • I should say container group or ACI instance initially. We have just 1 container inside of the container group - it's kind of equivalent for us in that case. So we have /29 subnet with 3 container groups/ACI instances. Regarding K8s discussion, it was suggested by a few folks from their support. We can't really count with "static IP" behavior with ACI, K8s services naturally fit there. On the other hand, also Azure support initially suggested a solution with /29 and 3 ACIs there to mitigate dynamic IP behavior :) You can choose :) But from my side, we should go with K8s from begining – O1da Feb 23 '21 at 14:37
  • And don't try to hack ACI. – O1da Feb 23 '21 at 14:37
1

I have implemented the following

  • Azure Alert that monitors the ACI Restart event
  • Triggers an Azure Function
  • Azure Function keeps Azure Private DNS up-to-date with latest IP

The function calls the API and get the new IP, then updates DNS. I have a short lived TTL on private DNS. The zone is only within my VPN.

This is not a perfect solution as this can mean 5 mins of downtime. However, I also have Azure Application Gateway and 3 instances. It's unlikely that all three instances would restart at the same time, and if they did, downtime would be inevitable.

Garry Taylor
  • 940
  • 8
  • 19