Azure Pipeline - Passing Variables between tasks

Question

All,

I have the below Azure DevOps pipeline setup that copy keyvault secrets from one KV to another. As you can see, I have two tasks: 1) one to read the secrets and 2) one to write the secrets. I am having difficulties figuring out how to pass the "$secrets" variable (thru "echo "##vso[task.setVariable variable=sourceSecrets]$json") from the first task to the second task.

stages:
  - stage: "Test1"
    displayName: "Test1 - Copy KV"
    jobs:
    - deployment : "Deploy"
      timeoutInMinutes: 120
      variables:
        sourceSecrets: ""
      strategy:
          runOnce:
            deploy:
              steps:
              - task: AzureCLI@2
                inputs:
                  azureSubscription: $(ServiceConnection1)
                  scriptType: 'pscore'
                  scriptLocation: 'inlineScript'
                  inlineScript: |
                          if ("$(mysubscription1)"){
                              az account set --subscription "mysubscription1"
                          }

                          $secNames = az keyvault secret list --vault-name "kvName1"  -o json --query "[].name"  | ConvertFrom-Json

                          Write-Host 'Reading secrets...'
                          $secrets = $secNames | % {
                              $secret = az keyvault secret show --name $_ --vault-name "kvName1" -o json | ConvertFrom-Json
                              [PSCustomObject]@{
                                  name  = $_;
                                  value = $secret.value;
                              } 
                          }
                          $json = $($secrets | ConvertTo-Json)
                          echo "##vso[task.setVariable variable=sourceSecrets]$json"
                          
              - task: AzureCLI@2
                inputs:
                  azureSubscription: $(ServiceConnection2)
                  scriptType: 'pscore'
                  scriptLocation: 'inlineScript'
                  inlineScript: |
                          if ("$(mysubscription2)"){
                              az account set --subscription $(mysubscription2)
                          }
                          $secrets = "$(sourceSecrets)" | ConvertFrom-Json
                          $secrets.foreach{
                          Write-Host 'Writing secrets:'
                              az keyvault secret set --vault-name $(kvName2) --name $_.name  --value  $_.value --output none
                              Write-Host '---->' $_.name
                          }

When the pipeline executes, tasks one executes fine. However, the 2nd task errored out with the following:

ConvertFrom-Json : Conversion from JSON failed with error: Error reading JArray from JsonReader. Path '', line 1, position 1.
At /home/vsts/work/_temp/azureclitaskscript1620360635888_inlinescript.ps1:4 char:18
+ $secrets = "[" | ConvertFrom-Json
+                  ~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [ConvertFrom-Json], ArgumentException
+ FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.ConvertFromJsonCommand

I did some checking, it appears the $(sourceSecrets) variable contain only "[" instead of the entire json content. This means the "echo "##vso[task.setVariable variable=sourceSecrets]$json" line from the first task is excluding everything after "[". I can't figure out why it is doing that. Ideas?

Thanks in advance.

Maybe this will help? https://stackoverflow.com/questions/55617951/how-do-i-copy-over-all-secrets-from-one-azure-keyvault-to-another-using-powershe — Repcak, May 07 '21 at 07:27
Ended up taking another approach and fixed the issue. https://stackoverflow.com/questions/46306467/powershell-json-formatting — Ultra GC, May 07 '21 at 23:22

Bright Ran-MSFT · Answer 1 · 2021-05-07T07:45:32.897

Generally, the value of the pipeline variable only supports string type, and should be a single line string. If you pass a multi-line content to a pipeline variable, normally only the first line will be received as the value of the variable.

In your case, the value you pass to the variable is a JSON object that contains multi-line content.

To avoid the issue you are facing, you should convert the content of the JSON object to be a single line string before passing it to the pipeline variable.

To convert a multi-line string to be a single line string, you can try the command lines below:

. . .

# escape '%', '\n' and '\r'
json="${json//'%'/'%25'}"
json="${json//$'\n'/'%0A'}"
json="${json//$'\r'/'%0D'}"

echo "##vso[task.setVariable variable=sourceSecrets]$json"

Hi Bright Ran. Thanks for the comment. Not familiar with the commands you've mention. I've tried the below but the variable is empty. Can you please clarify? Thanks ``` $json="${$json//'%'/'%25'}" $json="${$json//$'\n'/'%0A'}" $json="${$json//$'\r'/'%0D'}" ``` — Ultra GC, May 07 '21 at 19:40

Azure Pipeline - Passing Variables between tasks

1 Answers1