0

I have 2 resource groups as follow:

rg-shared
rg-storage-accounts

in resource group one I am trying to create a storage account and get its connection string and pass it to resourcegroup2 on which I have a key vault.

my actual code is as follow.

Shared.bicep

targetScope = 'resourceGroup'
param deploymentIdOne string = newGuid()
param deploymentIdTwo string = newGuid()
output deploymentIdOne string = '${deploymentIdOne}-${deploymentIdTwo}'
output deploymentIdTwo string = deploymentIdTwo

param keyvaultmain string = 'Name-keyvault'
param keyvaultshared string = 'Name-keyvault'
param sharedManagedIdentity string = 'Name-Managed-identity'
param storageAccountString string
var storagePrefix = 'sttesteur'
var clientDataKeyPrefix = 'Key-Data-'
var learnersguidsecrets = 'Guidtest'
param tenantCodes array = [
  'tste'
]

resource keyVaultClients 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {
  name: keyvaultmain
}

resource keyVaultShared 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {
  name: keyvaultshared
}

resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2018-11-30' = {
  name: sharedManagedIdentity
  location: resourceGroup().location
}

resource kvClientsKey 'Microsoft.KeyVault/vaults/keys@2021-06-01-preview' = [for code in tenantCodes: {
  name: '${keyVaultClients.name}/${clientDataKeyPrefix}${toUpper(code)}'
  properties: {
    keySize: 2048
    kty: 'RSA'
    // Assign the least permission
    keyOps: [
      'unwrapKey'
      'wrapKey'
    ]
  }
}]

resource accessPolicy 'Microsoft.KeyVault/vaults/accessPolicies@2021-06-01-preview' = {
  name: '${keyVaultClients.name}/add'
  properties: {
    accessPolicies: [
      {
        tenantId: subscription().tenantId
        objectId: managedIdentity.properties.principalId
        permissions: {
          // minimum required permission
          keys: [
            'get'
            'unwrapKey'
            'wrapKey'
          ]
        } 
      }
    ]
  }
}

resource clientLearnersGuid 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
  name: '${keyVaultClients.name}/${tenant}${learnersguidsecrets}'
  properties: {
    contentType: 'GUID Key'
    value: '${deploymentIdOne}-${deploymentIdTwo}'
  }
}]

resource storageAccountConnectionString 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
  name: '${keyVaultShared.name}${storagePrefix}${tenant}'
  properties:{
    contentType: '${tenant} Storage Account Connection String'
    value: storageAccountString
  }
}]

And this is my storage-account.bicep

param tenantCodes array = [
  'tste'
]

param tenantManagedIdentity string = 'Manage-identity-Name'
param secondresource string = 'rg-sec-eur-shared'
var keyVaultKeyPrefix = 'Key-Data-'
var storagePrefix = 'sthritesteur'






// Create a managed identity
resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2018-11-30' = {
  name: tenantManagedIdentity
  location: resourceGroup().location
}



// Create storage accounts
resource storageAccount 'Microsoft.Storage/storageAccounts@2021-04-01' = [for tenantCode in tenantCodes: {
  name: '${storagePrefix}${tenantCode}'
  location: resourceGroup().location
  kind: 'StorageV2'
  sku: {
    name: 'Standard_RAGRS'
  }
  // Assign the identity
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '${managedIdentity.id}': {}
    }
  }
  properties: {
    allowCrossTenantReplication: true
    minimumTlsVersion: 'TLS1_2'
    allowBlobPublicAccess: false
    allowSharedKeyAccess: true
    networkAcls: {
      bypass: 'AzureServices'
      virtualNetworkRules: []
      ipRules: []
      defaultAction: 'Allow'
    }
    supportsHttpsTrafficOnly: true
    encryption: {
      identity: {
        // specify which identity to use
        userAssignedIdentity: managedIdentity.id
      }
      keySource: 'Microsoft.Keyvault'
      keyvaultproperties: {
        keyname: '${keyVaultKeyPrefix}${toUpper(tenantCode)}'
        // keyvaulturi: keyVault.properties.vaultUri
        keyvaulturi:'https://keyvaultclient.vault.azure.net'
      }
      services: {
        file: {
          keyType: 'Account'
          enabled: true
        }
        blob: {
          keyType: 'Account'
          enabled: true
        }
      }
    }
    accessTier: 'Hot'
  }

}]

resource storage_Accounts_name_default 'Microsoft.Storage/storageAccounts/blobServices@2021-04-01' = [ for (storageName, i) in tenantCodes :{
  parent: storageAccount[i]
  name: 'default'
  properties: {
    changeFeed: {
      enabled: false
    }
    restorePolicy: {
      enabled: false
    }
    containerDeleteRetentionPolicy: {
      enabled: true
      days: 7
    }
    cors: {
      corsRules: []
    }
    deleteRetentionPolicy: {
      enabled: true
      days: 30
    }
    isVersioningEnabled: true
  }
}]

module connectionString 'shared.bicep' = [for (storageName, i) in tenantCodes :{
  scope: resourceGroup(secondresource)
  name: storageName
  params: {
    storageAccountString: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount[i].name};AccountKey=${listKeys(storageAccount[i].id, storageAccount[i].apiVersion).keys[0].value};EndpointSuffix=${environment().suffixes.storage}'
  }
}]

This is the details of this workflow.

in the resource group rg-sharedi have 2 key vaults, keyvault-sharedand keyvaultstorage. And their purpose is as follow:

keyvault-shared => Store StorageAccount Connection String as Secret

keyvault-storage => Generate a Key Name based on the `tenantCode` in the key section, and in secret, generate a GUID and store it

while in the other resource-group rg-storage I want to create a storage account, encrypt the storage account with the key I have generated in the keyault earlier, and pass the connection string of this storageAccount to the shared key vault.

Following your advice, I used the module from shared.bicep and called it in my storage account.bicep.

Based on my command:

az deployment group what-if -f ./storage-account.bicep -g rg-storage-accounts     

the output It shows that will create only the resource in the storage-account.bicep:

  • user identity
  • storageAccount
  • container

How to reproduce:

  • Create 2 resource groups (shared and storage accounts)
  • in Shared create 2 key vaults
  • Update the bicep files with the correct key vault names.
  • In both bicep scripts in tenantCode put a random name to create a storage account or multiple storage accounts.
  • and run the above bicep command.

I tried to explain as clear as I could this issue, as its driving me crazy and have no idea what I am doing wrong and this stage.

Please please, if you need anymore information about this issue, just ask and will be glad to clarify any doubt

UPDATE: To generate the key before hand, I moved the key creation into the storage.bicep as follow:

resource keyVaultClients 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {
  name: keyvaultmain
  scope: resourceGroup(secondresource)
}

resource kvClientsKey 'Microsoft.KeyVault/vaults/keys@2021-06-01-preview' = [for code in tenantCodes: {
  name: '${keyVaultClients.name}-${clientDataKeyPrefix}${toUpper(code)}'
  properties: {
    keySize: 2048
    kty: 'RSA'
    // Assign the least permission
    keyOps: [
      'unwrapKey'
      'wrapKey'
    ]
  }
}]

but I get this error:

{"error":{"code":"InvalidTemplate","message":"Deployment template validation failed: 'The template resource 'keyvault-Key-Data-ORNX' for type 'Microsoft.KeyVault/vaults/keys' at line '54' and column '46' has incorrect segment lengths. A nested resource type must have identical number of segments as its resource name. A root resource type must have segment length one greater than its resource name. Please see https://aka.ms/arm-template/#resources for usage details.'.","additionalInfo":[{"type":"TemplateViolation","info":{"lineNumber":54,"linePosition":46,"path":"properties.template.resources[1].type"}}]}}

Which I don't understand exactly to what refers.

UPDATE: This is an interesting output. So according to the last update (and thank you so so much for your help) I realised that at the code is creating all the correct resource, but at the very end it throws this error:

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentFailed\",\r\n        \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n        \"details\": [\r\n          {\r\n            \"code\": \"Conflict\",\r\n            \"message\": \"{\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"StorageAccountOperationInProgress\\\",\\r\\n    \\\"message\\\": \\\"An operation is currently performing on this storage account that requires exclusive access.\\\"\\r\\n  }\\r\\n}\"\r\n          },\r\n          {\r\n            \"code\": \"Conflict\",\r\n            \"message\": \"{\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"StorageAccountOperationInProgress\\\",\\r\\n    \\\"message\\\": \\\"An operation is currently performing on this storage account that requires exclusive access.\\\"\\r\\n  }\\r\\n}\"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"}]}}
Nayden Van
  • 1,133
  • 1
  • 23
  • 70
  • Does this answer your question? [Azure Bicep multiple scopes in template](https://stackoverflow.com/questions/69696317/azure-bicep-multiple-scopes-in-template) – Thomas Oct 25 '21 at 18:53
  • No it doesn't fully. Still getting the same error as mentioned in the op unfortunately – Nayden Van Oct 25 '21 at 18:57
  • @AnsumanBal-MT thank you so much for your time and reply. I am currently working on your suggestion. Shortly I will be able to post a reply with an answer – Nayden Van Oct 26 '21 at 09:37
  • @AnsumanBal-MT I did updated my OP with the full script as is not working. I tried to be as clear as possible. Thank you so much for your time and patience with a beginner. I am just trying to learn so I can understand how bicep works vs terraform – Nayden Van Oct 26 '21 at 10:33
  • hello @NaydenVan, please check the update i have made in the answer and let me know if it resolves you issue. – Ansuman Bal Oct 26 '21 at 11:56
  • @NaydenVan, for your case i guess you have to create 2 nested blocks then , 1st file will be ran in shared folder for keyvault clients then it will have nested file which will create a storage account and then again nested to store the connection string in the keyvault. – Ansuman Bal Oct 26 '21 at 13:23
  • @NaydenVan,sorry took me some time to get back here,, after lots of testing , updated the answer which is exactly as per your requirement. – Ansuman Bal Oct 26 '21 at 15:38
  • @AnsumanBal-MT thank you so much for you infinite patience. I updated my post, Is this error familiar to you? – Nayden Van Oct 26 '21 at 15:57
  • This happens if in the tenantCode you try to create multiple storages – Nayden Van Oct 26 '21 at 16:07
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/238562/discussion-between-ansumanbal-mt-and-nayden-van). – Ansuman Bal Oct 26 '21 at 16:12

1 Answers1

2

For testing , I used nested template module for creating a single Storage account and then stored the connection string in the key vault present in the another resource group.

Scenario:

Keyvaultclient.bicep>>nested(storage.bicep)>>nested(shared.bicep)

Code:

Keyvaultclient.bicep:

param deploymentIdOne string = newGuid()
param deploymentIdTwo string = newGuid()
output deploymentIdOne string = '${deploymentIdOne}-${deploymentIdTwo}'
output deploymentIdTwo string = deploymentIdTwo

param storagerg string = 'rgnamewherestorageaccountistobecreated'
param sharedManagedIdentity string = 'identityforkeyvault'
param keyvaultmain string = 'keyvaultclienttes1234'
param tenantCodes array = [
  'tste'
]
var clientDataKeyPrefix = 'Key-Data-'

resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2018-11-30' = {
  name: sharedManagedIdentity
  location: resourceGroup().location
}

resource keyVaultClients 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {
  name: keyvaultmain
}
resource kvClientsKey 'Microsoft.KeyVault/vaults/keys@2021-06-01-preview' = [for code in tenantCodes: {
  parent:keyVaultClients
  name: '${keyVaultClients.name}-${clientDataKeyPrefix}${toUpper(code)}'
  properties: {
    keySize: 2048
    kty: 'RSA'
    // Assign the least permission
    keyOps: [
      'unwrapKey'
      'wrapKey'
    ]
  }
}]
resource accessPolicy 'Microsoft.KeyVault/vaults/accessPolicies@2021-06-01-preview' = {
  parent:keyVaultClients
  name: 'add'
  properties: {
    accessPolicies: [
      {
        tenantId: subscription().tenantId
        objectId: managedIdentity.properties.principalId
        permissions: {
          // minimum required permission
          keys: [
            'get'
            'unwrapKey'
            'wrapKey'
          ]
        } 
      }
    ]
  }
}
resource clientLearnersGuid 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
  parent:keyVaultClients
  name: '${keyVaultClients.name}${tenant}'
  properties: {
    contentType: 'GUID Key'
    value: '${deploymentIdOne}-${deploymentIdTwo}'
  }
  dependsOn:kvClientsKey
}]
module StorageAccount './storage.bicep' = [for (storageName, i) in tenantCodes :{
  scope: resourceGroup(storagerg)
  name: storageName
  params: {
    ManagedIdentityid:managedIdentity.id
    kvname:keyVaultClients.name
    uri:keyVaultClients.properties.vaultUri
  }
  dependsOn:clientLearnersGuid
}]

Storage.bicep:

param tenantCodes array = [
  'tste'
]
param ManagedIdentityid string
param uri string 
param kvname string
param keyvaultrg string = 'rgwherethekeyvaultsarepresent'
var keyVaultKeyPrefix = 'Key-Data-'
var storagePrefix = 'sthritesteur'


// Create storage accounts
resource storageAccount 'Microsoft.Storage/storageAccounts@2021-04-01' = [for tenantCode in tenantCodes: {
  name: '${storagePrefix}${tenantCode}'
  location: resourceGroup().location
  kind: 'StorageV2'
  sku: {
    name: 'Standard_RAGRS'
  }
  // Assign the identity
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
        '${ManagedIdentityid}':{}
    }
  }
  properties: {
    allowCrossTenantReplication: true
    minimumTlsVersion: 'TLS1_2'
    allowBlobPublicAccess: false
    allowSharedKeyAccess: true
    networkAcls: {
      bypass: 'AzureServices'
      virtualNetworkRules: []
      ipRules: []
      defaultAction: 'Allow'
    }
    supportsHttpsTrafficOnly: true
    encryption: {
      identity: {
        // specify which identity to use
        userAssignedIdentity: ManagedIdentityid
      }
      keySource: 'Microsoft.Keyvault'
      keyvaultproperties: {
        keyname: '${kvname}-${keyVaultKeyPrefix}${toUpper(tenantCode)}'
        keyvaulturi:uri
      }
      services: {
        file: {
          keyType: 'Account'
          enabled: true
        }
        blob: {
          keyType: 'Account'
          enabled: true
        }
      }
    }
    accessTier: 'Hot'
  }

}]



resource storage_Accounts_name_default 'Microsoft.Storage/storageAccounts/blobServices@2021-04-01' = [ for (storageName, i) in tenantCodes :{
  parent: storageAccount[i]
  name: 'default'
  properties: {
    changeFeed: {
      enabled: false
    }
    restorePolicy: {
      enabled: false
    }
    containerDeleteRetentionPolicy: {
      enabled: true
      days: 7
    }
    cors: {
      corsRules: []
    }
    deleteRetentionPolicy: {
      enabled: true
      days: 30
    }
    isVersioningEnabled: true
  }
}]

module connectionString './shared.bicep' = [for (storageName, i) in tenantCodes :{
  scope: resourceGroup(keyvaultrg)
  name: storageName
  params: {
    storageAccountString: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount[i].name};AccountKey=${listKeys(storageAccount[i].id, storageAccount[i].apiVersion).keys[0].value};EndpointSuffix=${environment().suffixes.storage}'
  }
}]

shared.bicep:

param keyvaultshared string = 'keyvaultsharedtest12345'
param storageAccountString string
param tenantCodes array = [
  'tste'
]
resource keyVaultShared 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {
  name: keyvaultshared
}
resource storageAccountConnectionString 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
  parent:keyVaultShared
  name: '${keyVaultShared.name}-test${tenant}'
  properties:{
    contentType: '${tenant} Storage Account Connection String'
    value: storageAccountString
  }
}]

Output:

keyvaultclient.bicep will be deployed to the kvresourcegroup:

az deployment group create -n TestDeployment -g keyvaultrg --template-file "path\to\keyvaultclient.bicep"

enter image description here

enter image description here

enter image description here

enter image description here

Ansuman Bal
  • 9,705
  • 2
  • 10
  • 27
  • thank you for your help. The code works fine but I saw you removed the encryption side. The storage account should be encrypted with the key that has been created – Nayden Van Oct 26 '21 at 12:11
  • I am testing that, but the script is still running. That bit is really important for me to understand the whole workflow and bicep structure. – Nayden Van Oct 26 '21 at 12:20
  • So I added the encryption to your code. and run the test against one storage account and that worked because the key existed already. As second test, in tenantCodes I added 2 storages (new) and run the script and failed with the error 'key vault/key' not found. Because is trying to retrieve a key that does not exist yet – Nayden Van Oct 26 '21 at 12:36
  • thank you so much for your patience and your kindness and I am really sorry if I am annoying. But the std error for bicep for me is really hard to understand as it does not point to any clear error. I updated my post with the latest error If you can help me to understand it – Nayden Van Oct 26 '21 at 12:55
  • I am so confused now. I added `parent:keyVaultClients` but it won't allow me because on the parent I am declaring the scope, which I have to. – Nayden Van Oct 26 '21 at 13:27
  • @NaydenVan ,I am doing a final test with encryption and all let me get back to you in few mins – Ansuman Bal Oct 26 '21 at 13:38
  • Thank you so so so much – Nayden Van Oct 26 '21 at 14:19