Why is powershell -replace operator not properly including all characters captured in the capture group?

Question

Edit:

After reviewing this question again, I've added a second example that I hope will highlight where my confusion became amplified, namely there was one capture group accessed by its index (1), and the value I expected from the $filecontent was coincidentally also a 1.

---

This question indicates that a back tick can be used to address capture groups in a double quoted string when referencing other variables.

If you need to reference other variables in your replacement expression (as you may), you can use a double-quoted string and escape the capture dollars with a backtick

However, I'm seeing some interesting behavior that I can't explain.

$VersionReplacementRegex = "(\d+\.)\d+" #capture first digit + dot b/c I want to keep it
$BuildVersionValidationRegex = "\d+\.\d+\.\d+"
    
$VersionData = [regex]::matches("some-18.11.8",$BuildVersionValidationRegex)
$NewVersion = $VersionData[0] #matches 18.11.8

$filecontent = "stuff 1.0.0.0 other stuff" #Get-Content($file)

replacing text in $filecontent using the capture group as specified in the linked question gives an incomplete result...

$filecontent -replace $VersionReplacementRegex, "`$1$NewVersion" | Write-Host

returns: 118.11.8 expected: 1.18.11.8

But adding a space between the $1 and $NewVersion gives a different yet equally unhelpful result..

$filecontent -replace $VersionReplacementRegex, "`$1 $NewVersion" | Write-Host

returns: 1. 18.11.8 The captured dot appears here, but so does the undesirable space.

With this example, the results are somewhat similar, but it seems that the capture group is getting the wrong value all together.

$NewVersion = 18.11.8
$filecontent = "stuff 5.0.0.0 other stuff"
$filecontent -replace "(\d+\.)\d+", "`$1$NewVersion" | Write-Host

# returns: 118.11.8
# expected: 5.18.11.8

Adding the space in the replacement string returns: 5. 18.11.8

So, what am I missing, or is there a better way to do this?

Answer 1

^{Judging from past experience, PetSerAl, who provided the crucial pointer in a comment on the question, won't be back to post an answer.}

tl;dr

If you use -replace with a replacement operand that references capture groups and PowerShell variables, use syntax such as "`${<ndx>}${<PsVar>}", where <ndx> is the index of your capture group, and <PsVar> is the name of your PowerShell variable; note the ` before the first $:

PS> $var = '2'; 'foo' -replace '(f)', "[`${1}$var]"
[f2]oo # OK, -replace saw '${1}2'

If you neglect to use {...} to disambiguate the capture-group index, the replacement malfunctions, because the interpolated string value then effectively references a different index:
-replace then sees [$12], which, due to referring to a nonexistent capture group with index 12, is left as-is:

PS> $var = '2'; 'foo' -replace '(f)', "[`$1$var]"
[$12]oo # !! -replace saw '$12', i.e., a nonexistent group with index 12

---

It is tricky to mix PowerShell's string expansion (interpolation) with the syntax of the
-replace operator, because it is easy to get confused:

• In double-quoted ("...") strings, it is PowerShell's generic string expansion (string interpolation) feature that interprets $ chars first, where a $ prefix refers to (PowerShell) variables and, inside $(...), entire statements.

• Whatever string is the result of that expansion is then interpreted by the -replace operator, where $-prefixed tokens refer to results of the regex-matching operation, as summarized in this answer.

• Note that these layers of $ interpretation are entire unrelated and the fact that both use sigil $ is incidental.

Therefore:

• If your replacement operand doesn't need string expansion, i.e., if there's no need to reference PowerShell variables or expressions, be sure to use a single-quoted string ('...'), so that PowerShell's string expansion doesn't come into play:

   PS> 'foo' -replace '(f)', '[$1]'
   [f]oo  # OK - if you had used "[$1]" instead, the output would be '[]oo',
          # because $1 is then interpreted as a *PowerShell variable*.
  

• If you do need to involve string expansion:

  • Prefix $ chars. that should be passed through to -replace with `

    • ` (the backtick) is PowerShell's general escape character, and in "..." strings it is used to indicate that the next character is to be taken literally; placed before a $, it suppresses string interpolation for that token; e.g., "I'm `$HOME" yields literal I'm $HOME, i.e., the variable reference was not expanded.

  • To disambiguate references to capture groups, e.g., $1, enclose them in {...} - e.g., ${1}

    • Note that you may also need to use {...} to disambiguate PowerShell variable names; e.g. "$HOME1" must be "${HOME}1" in order to reference variable $HOME successfully.
    • Also, it is not just about capture-group indices; ambiguity can also arise with named capture groups; in "..."-based replacement operands, always using {...} around capture-group indices / names (and PS variables) is a good habit to form.

  • If in doubt, output the replacement operand by itself in order to inspect what -replace will ultimately see.

    • In the example above, outputting "[`$1$var]" by itself, which applies the string-interpolation step, would have made the problem more obvious: [$12]

To illustrate the latter point:

PS> $var = '2'; 'foo' -replace '(f)', "[`$1$var]"
[$12]oo  # !! $1 wasn't recognizes as the 1st capture group.

The problem was that -replace, after string expansion, saw [$12] as the replacement operand, and since there was no capture group with index 12, it was left as-is.

Enclosing the capture-group number in {...} solves the problem:

PS> $var = '2'; 'foo' -replace '(f)', "[`${1}$var]"
[f2]oo  # OK