Why is Powershell output different between 5.1 and Core 7.1.4

Question

I have a block of PowerShell code that runs a command and then massages it and writes it to a text file.

$commandOutput = &"C:\Program Files (x86)\Dell EMC\Unity\Unisphere CLI\uemcli" -d $frame -noHeader /stor/config/pool/sr show
foreach ($idline in $commandOutput)
{
    if ($idline -match ' ID ')
    {
    $subline = $idline.substring(6,($idline.length - 6))
    $writeline = $subline.trim()
    #$writeline
    }
    elseif ($idline -match 'Total pool space used')
    {
    $Sleft,$Sright = $idline.Split('(',2)
    $writeline = $Sleft.trim()
    #    $writeline
    #    write-host ""
    } 
 
    else {$writeline = $idline.trim()}
    #$writeline
    Add-Content -path $outdir\$frame-intrim.txt -value $writeline -Encoding UTF8
    }

When I run it in PowerShell 5.1 I get:

ID                    = res_2
Name                  = asi-vcva-fs01
Resource type         = VMware NFS
Pool                  = pool_1
Total pool space used = 18025199894528
Health state          = OK (5)

When I run it in PowerShell Core 7.1.4 I get:

1 :          I D                                         =   r e s _ 2 
             N a m e                                     =   a s i - v c v a - f s 0 1 
             R e s o u r c e   t y p e                   =   V M w a r e   N F S 
             P o o l                                     =   p o o l _ 1 
             T o t a l   p o o l   s p a c e   u s e d   =   1 8 0 2 5 2 1 3 2 7 2 0 6 4   ( 1 6 . 3 T ) 
             H e a l t h   s t a t e                     =   O K   ( 5 )

Which messes up the rest of the script. I tried -Encoding UTF8 but nothing changed. How do I make this able to run in 5.1 as well as Core 7.1?

More Information:

The reason it is writing the output that way is because that is the way the data is parsed out of the input. How do I get rid of the spaces especially to make it usable in both versions of code?

mklement0 · Accepted Answer · 2021-08-29T22:09:33.500

I don't know why the behavior differs between the PowerShell editions in this case, but it looks like uemcli outputs UTF-16LE ("Unicode") encoded strings.

For PowerShell to recognize them correctly^[1], (temporarily) set
[Console]::OutputEncoding = [Text.Encoding]::Unicode
before invocation.

As for diagnosing the character encoding used by a particular external program and how PowerShell decodes it:

See the bottom section of this answer, which provides two helper functions:
- Debug-NativeInOutput helps you diagnose in- and output to and from external programs.
- Invoke-WithEncoding makes external-program invocations with a given encoding easier; e.g., in your case you would call:
```
Invoke-WithEncoding -Encoding Unicode { 
  & "C:\Program Files (x86)\Dell EMC\Unity\Unisphere CLI\uemcli" -d $frame -noHeader /stor/config/pool/sr show
}
```
This answer contains another helper function, Debug-String, which can help you find non-printing and non-ASCII characters in strings in general; e.g., ASCII-range characters that were misdecoded from UTF-16LE output would result in that character plus a NUL (0x0) character, which Debug-String would surface as `0.
- By default, NUL characters print like spaces in the console, which is what you saw.

^{[1] in subsequent processing; note that direct-to-display output may print fine, but character encoding issues may surface as soon as PowerShell decodes the output, which happens invariably in the course of capturing output in a variable or sending it through the pipeline or to a file.}

Why is Powershell output different between 5.1 and Core 7.1.4

1 Answers1