1

I'm trying to execute such command in a Powershell script:

Write-Output "Some Command" | some-application

When running PS script some-application receives string \xef\xbb\xbfSome Command. The first character is an UTF-8 BOM. All solutions that I can Google, apply only to redirecting output to a file. But I'm trying to redirect a string to another command (via pipe).

The variable $OutputEncoding shows that ASCII is configured, no UTF-8 is set.

I'm running this script from Azure DevOps Pipeline and only there this problem exists.

Rython
  • 577
  • 9
  • 17
  • Perhaps [this](https://stackoverflow.com/questions/40098771/changing-powershells-default-output-encoding-to-utf-8) can help you? – Theo Nov 19 '20 at 13:31
  • Does this answer your question? [Using PowerShell to write a file in UTF-8 without the BOM](https://stackoverflow.com/questions/5596982/using-powershell-to-write-a-file-in-utf-8-without-the-bom) – Nico Nekoru Nov 19 '20 at 15:54

1 Answers1

0

Note: This answer deals with how to control the encoding that PowerShell uses when data is piped to an external program (to be read via stdin by such a program).

This is separate from:

  • what character encoding PowerShell cmdlets use by default on output - for more information, see this answer.

  • what character encoding PowerShell uses when reading data received from external programs - for more information, see this answer.

The implication is that you've mistakenly set the $OutputEncoding preference variable, which determines what character encoding PowerShell uses to send data to external programs, to a UTF-8 encoding with a BOM.

Your problem goes away if you assign a BOM-less UTF-8 encoding instead:

$OutputEncoding = [System.Text.Utf8Encoding]::new($false) # BOM-less

"Some Command" | some-application

Note that this problem wouldn't arise in PowerShell [Core] v6+, where $OutputEncoding defaults to BOM-less UTF-8 (in Windows PowerShell, unfortunately, it defaults to ASCII).

To illustrate the problem:

$OutputEncoding = [System.Text.Encoding]::Utf8 # *with* BOM

"Some Command" | findstr .

The above outputs Some Command, where  is the rendering of the 3-byte UTF-8 BOM (0xef, 0xbb, 0xbf) in the OEM code page 437 (on US-English systems, for instance).

mklement0
  • 382,024
  • 64
  • 607
  • 775