Escape double ampersand in powershell script when using Invoke-Expression

Question

I am trying to build a command using different strings that are created conditionally. This command is for adding a key to vault.

Thing is, either in key description or value double ampersands can occur.
I managed to escape single ampersand, but not the two or more.
Current code, excluding parameter passing etc for brevity:

[string]$params = "az keyvault secret set --name `'$name`' --vault-name `'$destinationVault`' "

    if($description -ne '')
    {
        # Escaping ampersand so that it can be used in Invoke-Expression
        $params += "--description `'$($description -replace '&', '"&"')`' "
    }

    $params += "--disabled $(if($enabled) { "false" } else { "true" }) "

    if($expires -ne '')
    {
        $params += "--expires `'$(([DateTime]($expires)).ToUniversalTime().ToString($dateFormat))`' "
    }

    if($notBefore -ne '')
    {
        $params += "--not-before `'$(([DateTime]($notBefore)).ToUniversalTime().ToString($dateFormat))`' "
    }

    if($tags -ne '')
    {
        $params += "--tags `'$tags`' "
    }
    else
    {
        $params += "--tags `"`" "    
    }

    $params += "--value `'$($value -replace '&', '"&"')`'"
    
    return $params

Returned params is used by parent script - $result = Invoke-Expression (Build-SetSecretCommand $secret $destinationKeyVault)

Result of this code ran on example data with single & in secret value in console looks like that:

az keyvault secret set --name 'secret_name' --vault-name 'vault_name' --disabled false --tags 'file-encoding=utf-8' --value 'super_secret://+_with_symbols"&"ampersand!'

and translates to super_secret://+_with_symbols&ampersand! in created secret value.

As mentioned, code works with single &, but if there are more than one symbols like that, I am getting errors like

Invoke-Expression : At line:1 char:23
+ Write-Host asdasdasd a&&&&&asdASsdad asd a    ada   &7
The token '&&' is not a valid statement separator in this version.

Answer 1

First, the obligatory warning:

• Invoke-Expression (iex) should generally be avoided and used only as a last resort, due to its inherent security risks. Superior alternatives are usually available. If there truly is no alternative, only ever use it on input you either provided yourself or fully trust - see this answer.

In the case at hand, since you're calling an external program (az), the best approach is to create to collect the programmatically constructed arguments in an array, as the following simplified example demonstrates:

# Programmatically construct the arguments to pass to az below.
$azArgs = & {

  if ($description) 
  {
      '--description'
      # Enclose (runs of) ampersands (&) in arguments with *no spaces* in "..."
      # THIS MUST BE DONE FOR ALL ARGUMENTS THAT POTENTIALLY CONTAIN & 
      # such as the --value argument.
      # IF AN ARGUMENT (ALREADY) HAS EMBEDDED " CHARACTERS, MORE WORK IS NEEDED, 
      # EVEN IF IT CONTAINS SPACES - see notes below.
      if ($description -match '&' -and $description -notmatch ' ') {
        $description -replace '&+', '"$&"' 
        # NOTE: $& is a placeholder for whatever the regex matched
        #       That this placeholder also includes '&' is coincidental.
      } else {
        $description
      }
  }

  '--disabled'
  if ($enabled) { 'false' } else { 'true' }

  # ...
}

# Invoke az with the array of arguments constructed above.
az keyvault secret set --name $name --vault-name $destinationVault $azArgs

Note:

• The script block ({ ... }) outputs individual strings, which PowerShell automatically collects in an array (assuming two or more outputs).

• & chars. embedded in arguments (potentially along with other cmd.exe metacharacters such as |) only cause problems if the argument contains no spaces; this unfortunate problem is a confluence of two behaviors:

  • PowerShell - justifiably - passes space-less arguments unquoted on the command line it rebuilds behind the scenes - irrespective of the original quoting; e.g., 'a&b' is placed as unquoted a&b on the command line ultimately used for invocation.

  • az - due to being implemented as a batch file (az.cmd) - regrettably parses its command line as if it had been invoked from inside a cmd.exe session, so that an unquoted argument such as a&b breaks the call, because & is a cmd.exe metacharacter. (The problem does not arise if the argument happens to contain spaces, because PowerShell then places it double-quoted on the behind-the-scenes command line.)

  • While cmd.exe is ultimately at fault, PowerShell could - and arguably should - compensate for this automatically, as proposed in GitHub issue #15143. Sadly, it appears that this proposal won't be implemented.

• Applying embedded double-quoting to (runs of) & chars. in space-less arguments is not enough if arguments already have embedded double quotes ("), e.g. a"b; in fact, preexisting embedded " are a problem even in arguments with spaces. They require explicit escaping as "", which - in contrast with the & issue - is PowerShell's fault: passing arguments with embedded " has always been broken and still is as of PowerShell 7.2.1 - a possibly opt-in fix may come in v7.3 at the earliest - see this answer

A backward- and forward-compatible helper function that obviates the need for manual quoting and escaping is the ie function from the Native module (Install-Module Native)

With the ie function installed, you can output the argument values as-is from the script block (e.g., if ($description) { '--description'; $description }) and simply prepend ie to the invocation:

# Call via helper function "ie" from the "Native" module
# - no manual quoting / escaping needed.
ie az keyvault secret set --name $name --vault-name $destinationVault $azArgs

Answer 2

After analyzing and applying bulk of knowledge from mklement0 excellent post, I wrote a small utility to fill my needs - to escape ", &, | and any combination of such characters, mostly in connection strings, etc.

function ConvertTo-CmdSafeString
{
    [OutputType([string])]
    param (
        [Parameter()]
        [string]$Source
    )
    if($Source -notmatch ' ')
    {
        $Source = $Source -replace '[\"]', '"$&'
        $Source = $Source -replace '[\"\|&]{1,}', '"$&"'
    }

    return $Source
}

This, given the
$secret.value of

"abc"bca""abc"""bca""""abc&abc&&abc&&&abc""||&&"abc"|&&|"|"|"|"||abc||abc|||abc&|abc|&|&abc"&abc"&|&"abc&&||abc""'"&&||""abc"''

It converts it to

""""abc""""bca""""""abc""""""""bca""""""""""abc"&"abc"&&"abc"&&&"abc"""""||&&"""abc"""|&&|""|""|""|""||"abc"||"abc"|||"abc"&|"abc"|&|&"abc"""&"abc"""&|&"""abc"&&||"abc""""""'"""&&||"""""abc""""''

Which after executing $azsecret = (az $data) | ConvertFrom-Json
gives us $azsecret.value of

"abc"bca""abc"""bca""""abc&abc&&abc&&&abc""||&&"abc"|&&|"|"|"|"||abc||abc|||abc&|abc|&|&abc"&abc"&|&"abc&&||abc""'"&&||""abc"''

If you see some errors in it or missed edge-cases, please let me know.