0

In my current situation, we have several Azure Function apps that talk to an Azure SQL database (using Entity Framework, should it matter), using functions that trigger on an Azure ServiceBus trigger.

In the last weeks we have been improving security by using a VNET and subnets to only allow access to the Azure Database server by only the Function apps that need to use it. However, I have run into a strange issue. It seems that now the database server is set to disallow traffic apart from the defined subnets in which my fuction apps run, the function apps start giving intermittent SQLExceptions when connecting to the database with the message that some specific IP is not allowed by the database Firewall rules. The weird thing is that this error is not consistent. I would expect either for the function app to be declined at the firewall for it's IP, or be allowed all the time, but not randomly as is currently the case.

Question

Is there something that I am missing with my setup? Or, how do I force my function apps in subnets to use their internal IP that is allowed by the database server firewall rules, and not some other outbound IP address that is not in the database firewall rules? Alternatively: What can possibly explain that access to the database sometimes succeed (indicating a proper internal IP used by the funcation app), and sometimes fail on the firewall (with an unknown IP address), seemingly at random.

Hopefully somebody can help!

Detailed Description of situation

The Function App has a function that is triggered by a Service Bus trigger. The Function App is running with a P1v2 Premium service plan with Vnet integration on.

enter image description here

The app is running inside a Subnet in our environment with a defined IP adress range with a /26 subnet mask. If I check the environment variables of the funciton app in Kudu I can see the PRIVATE_IP_ADDRESS setting is in the subnet range. The database firewall is set up to disallow all traffic, apart from the subnet of my function app as follows:

enter image description here

Triggering the function app which will write some stuff into the database works sometimes (Indicating that the access to the database is working at least when the IP address of the function is the correct one) however, there are also a lot of SQLExceptions with the following error:

Cannot open server 'database-server-name' requested by the login. Client with IP address 'XXX.XXX.XXX.XXX' is not allowed to access the server. To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range. It may take up to five minutes for this change to take effect.

The IP address mentioned in the error is NOT the internal IP defined on the Vnet or subnet IP range. It is also not even one of the IP's that show up in the Possible Outbound Ip addresses of the function app,

These happen more or less randomly. Sometimes they don't appear for x triggers, sometimes it fails for hundres of calls in a row.

Enabling the Datbaase server setting "Allow Azure services and resources to access this database server" stops the error from occuring, but of course that is counter to configuring the firewall to allow certain subnets

What I have tried

Céryl Wiltink
  • 1,879
  • 11
  • 13

2 Answers2

1

Your Azure Function will only use a private outbound IP address picked from your VNet when you have VNet integration enabled. The VNet integration option is only available in the Function Premium Plan.

Additionally, the environment variables WEBSITE_VNET_ROUTE_ALL and WEBSITE_DNS_SERVER you already mentioned should be set as well as mentioned here (assuming your SQL server is in the same VNet).

Christian Vorhemus
  • 2,396
  • 1
  • 17
  • 29
  • Thank you. Yes, Vnet integration is enabled, Apps are running in a P1v2 Premium function plan (I see this as an answer a lot on the related questions, I should have mentioned it, but I'll add it). And as mentioned, I set the variables you mentioned. As SOME of the times the function app CAN access the database, that should be a clear indicator it is actually configured correctly correctly, but something else is wrong, no? – Céryl Wiltink Apr 02 '21 at 06:40
  • I don't get the "SOME of the times the function app CAN access the database" part - does your function always pick a private IP address? Or is the outbound IP a public one? – Christian Vorhemus Apr 02 '21 at 07:38
  • That's just it. It was sometimes the private IP, and sometimes the public IP, depending on a flip of the coin. When it was the public IP, the database did not allow it. It was more appearant under high traffic, so my current assumption is that there is some "scaling" going on, and during high traffic, some funciton code is run on physically different machines. This is just an assumption though and I selfanswered: what was needed was a quick switch between Standard and Premium service plans to force an IP reset. Thanks for the help anyway! – Céryl Wiltink Apr 02 '21 at 08:15
0

Of course a new day and fresh perspective brings the probable answer.

While I havent changed anything about the setup I mentioned in my question, I was reading this post again: https://learn.microsoft.com/en-us/azure/azure-functions/ip-addresses#outbound-ip-address-changes

It mentioned switching the service plan temporary force an IP change. I initially misinterpreted this as switching between the DEV and premium plans, which cannot happen because the DEV plans don't support VNET integration. So I switched plans between the P1v2 and P1v3 plans. This however did not work.

What was meant here was to switch beween S1 and P1v2 plans. The Standard plans are hidden behind this link:

enter image description here

It also shows this small message net to the Apply button.

enter image description here

After switching between the S1 and P1v2 plans for a moment and resubmitting deadlettered servicebus messages to the function app, everything started working again.

I assume the IP switch was necessary, but switching between P-plans is not what triggers it. It has to be between standard and premium.

enter image description here

Céryl Wiltink
  • 1,879
  • 11
  • 13