1

Is it possible to access Azure Key Vault Secrets with Azure Managed Identity setup on the Azure Batch? I've read through Microsoft docs and the only option I technically see is using certificate based authentication to access key vault within the .net console application. Since Managed Identities can be used in Azure Batch, I figured DefaultAzureCredential could be taken advantage of. I've had no luck in getting it to work.

Has anyone been able to get it to work like this, or am I going to have to settle for using certificate based authentication? Microsoft if you see this... Is there something in the works to get something like this working with Managed Identities

Technologies in play:

  • Azure Managed Identity: User-Assigned (id: xxxx)
  • Azure Key Vault
    • Added Managed Identity (id: xxx) to access policy with secrets access: get, list
  • Azure Batch
    • Batch Pool
      • Managed Identity (id: xxxx) Installed
      • Application: .Net Console Application (where code below resides)
  • .Net Console Application
    • .Net6

Here is what I am attempting to do:

DefaultAzureCredential credential = new DefaultAzureCredential( 
    new DefaultAzureCredentialOptions()
      {
         ManagedIdentityClientId = "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx"
      });
            
Uri uri = new("https://xxxxxxx.vault.azure.net/"); //Also Tried "https://xxxxxxx.vault.azure.net/secrets/"
            
SecretClient client = new(uri, credential);
var secret = await client.GetSecretAsync(secretName);
return secret?.Value.Value;

Error I receive from Batch Job's Task (filtered to show the ManagedIdentityCredential error)

Unhandled exception. Azure.Identity.CredentialUnavailableException: ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource.
Status: 400 (Bad Request)

Content:
{"error":"invalid_request","error_description":"Identity not found"}

Headers:
Server: IMDS/150.870.65.684
Date: Wed, 27 Jul 2022 17:47:07 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 68

   at Azure.Identity.ImdsManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, Response response, CancellationToken cancellationToken)
   at Azure.Identity.ManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
   at Azure.Identity.ImdsManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
   at Azure.Identity.ManagedIdentityClient.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
   at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage)
   at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
   at Azure.Identity.ManagedIdentityCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthenticateRequestAsync(HttpMessage message, Boolean async)
   at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.ProcessCoreAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted(ValueTask task)
   at Azure.Core.Pipeline.RedirectPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted(ValueTask task)
   at Azure.Core.Pipeline.RetryPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory`1 pipeline)
   at Azure.Core.Pipeline.HttpPipeline.Send(HttpMessage message, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.HttpPipeline.SendRequest(Request request, CancellationToken cancellationToken)
   at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest(Request request, CancellationToken cancellationToken)
   at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest[TResult](RequestMethod method, Func`1 resultFactory, CancellationToken cancellationToken, String[] path)
   at Azure.Security.KeyVault.Secrets.SecretClient.GetSecret(String name, String version, CancellationToken cancellationToken)
   at wileyulab.infrastructure.SimpleService.Execute(String[] args) in C:\AzDo\WileyU\wiley-u-lab\wileyulab.infrastructure\SimpleService.cs:line 55
   at wileyulab.consoleapp.simple.Program.Main(String[] args) in C:\AzDo\WileyU\wiley-u-lab\wileyulab.consoleapp.simple\Program.cs:line 70
A. Wiley
  • 11
  • 3
  • Do you have the proper `AZURE_TENANT_ID` set? – fpark Jul 28 '22 at 17:44
  • Have you configure the user-assigned identity correctly ? https://learn.microsoft.com/en-us/azure/batch/managed-identity-pools – Thomas Jul 28 '22 at 20:11
  • I have exactly the same issue doing the same steps/code - did you ever find a solution to this? – Ran Sagy Mar 06 '23 at 11:33
  • @RanSagy - you need to create a pool with the user created managed identity associated with it in order to use the identity from the Batch node – Gaurav Mar 07 '23 at 22:42
  • @Gaurav I did do that, I can it is associated in the portal. However, I still can't get the ManagedIdentity portion of DefaultAzureCredential to pick on it. I also have AZURE_TENANT_ID and AZURE_CLIENT_ID env vars set. the code also gives the DefaultAzureCredential initializer the value of AZURE_CLIENT_ID as the ManagedIdentity ID - the same one associated with the pool. – Ran Sagy Mar 08 '23 at 23:40
  • 1
    @RanSagy - are you sure you did it for the `pool` and not the `Batch account` itself? you have specifically create a pool with `User assigned identity` associated with the pool during creation. Once you do that, you can use the identity with `DefaultAzureCredentials` like you have mentioned: on C#: `new DefaultAzureCredential(new DefaultAzureCredentialOptions { ManagedIdentityClientId = }); ` I have done something similar with python sdk too – Gaurav Mar 10 '23 at 17:28
  • @Gaurav My path is pretty similar but a little bit different - I created the pool with the Azure CLI and/or Azure Powershell module, which can't give an identity during creation, and then HTTP PATCH'd it with an identity. Maybe something in that breaks the process, not sure. My .NET Code is exactly like what you've shown. – Ran Sagy Mar 11 '23 at 21:03
  • @RanSagy - can you validate if the identity is actually being assigned to the pool by looking at the properties blade of the pool via Azure Portal? There's a field/property called `User-assigned managed identities` in the properties of the pool. That should have your user assigned identity. – Gaurav Mar 13 '23 at 22:20
  • @Gaurav It is being assigned, yes. After a bit more trial and error, i figured out the difference in my scenario and it works now - I was creating the pool without the identity, as said (can't be done with the az CLI/Az powershell module yet, which is what i use in this scenario) but also with a target spot/low-prio node count of 1, meaning the machine was live before PATCH'ing in the identity happened. There's a restart to the nodes after all of that, but that wasn't enough apparently; the IMDS service still failed until i re-scaled to 0 and back to 1 to get a new machine, then it worked. – Ran Sagy Mar 15 '23 at 10:24
  • @RanSagy Wow. that spot vm was an edge case I didn't think of! Nice. Glad you figured it out. – Gaurav Mar 15 '23 at 18:51
  • @Gaurav I'm pretty sure a dedicated machine would act the same, just requires the identity configuration to exist before the machine enters the pool, it seems. Thanks for the tips to push me in the right direction though :) – Ran Sagy Mar 17 '23 at 16:57

1 Answers1

0

The error below may occur due to one of the following actions not properly configured.

Unhandled exception. Azure.Identity.CredentialUnavailableException: ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource.

  • Please make sure to ENABLE managed identity in the Azure service before using . enter image description here

  • As you have you user-asigned managed identity with the DefaultAzureCredential ,you need to provide the client Id and make sure specify the clientId in code by setting environment variables.

    AZURE_CLIENT_ID - service principal's app id
AZURE_TENANT_ID - principal's Azure Active Directory tenantId
AZURE_CLIENT_SECRET - service principal's client secrets

  • Make sure the managed identity is granted proper role like contributor or reader to access the batch and keyvault.

  • In the Azure portal, after the Key Vault is created,please add the Batch account access using managed identity in the Access Policy under Setting which is Under Key Permissions. select Get,list,create, Wrap Key and Unwrap Key etc which are needed

    enter image description here

Note:

And please Wait for at least 15 minutes for role to propagate and then try to access.

Please make sure you are logged in into Visual Studio within azure with proper roles/rights as Managed identity can work only when it is running in the Azure service. It will NOT work when run locally.

References:

  1. Configure customer-managed keys for your Azure Batch account with Azure Key Vault and Managed Identity - Azure Batch | Microsoft Docs
  2. TROUBLESHOOTING· GitHub
kavyaS
  • 8,026
  • 1
  • 7
  • 19