How to change the ps1 file in Unicode BigEndian to ASCII?

Question

Our automated build script will sign powershell scripts too. But Some of our powershell scripts are not signed . When i analyzed , we found that there is a known gotcha that Files saved by Powershell ise is saving in Unicode BigEndian which can't be signed.

As it is automated process, if a way to check whether a file is saved in Unicode big endian then change it to ASCII would solve our issue.

In powershell is there a way?

PowerShell ISE (verified in v3) creates _UTF-8_ files _with a BOM_ (Byte-Order Mark), not Big-Endian Unicode files. The BOM may be the problem, because the concept of a BOM technically doesn't apply to UTF-8, and its use is discouraged, but on Windows platforms it is used to mark files explicitly as UTF-8. — mklement0, Jan 21 '16 at 06:45
Note that if your source code contains non-ASCII chars., they will be replaced with literal `?` chars. on saving with `Out-File -Encoding ASCII`. Saving to a UTF-8 file _without_ a BOM is nontrivial, unfortunately, because `Out-File` doesn't support it - see http://stackoverflow.com/q/5596982/45375 — mklement0, Jan 21 '16 at 06:51

score 2 · Accepted Answer · answered Jan 21 '16 at 05:07

I found a function here that gets file encoding:

<#
.SYNOPSIS
Gets file encoding.
.DESCRIPTION
The Get-FileEncoding function determines encoding by looking at Byte Order Mark (BOM).
Based on port of C# code from http://www.west-wind.com/Weblog/posts/197245.aspx
.EXAMPLE
Get-ChildItem  *.ps1 | select FullName, @{n='Encoding';e={Get-FileEncoding $_.FullName}} | where {$_.Encoding -ne 'ASCII'}
This command gets ps1 files in current directory where encoding is not ASCII
.EXAMPLE
Get-ChildItem  *.ps1 | select FullName, @{n='Encoding';e={Get-FileEncoding $_.FullName}} | where {$_.Encoding -ne 'ASCII'} | foreach {(get-content $_.FullName) | set-content $_.FullName -Encoding ASCII}
Same as previous example but fixes encoding using set-content
#>
function Get-FileEncoding
{
    [CmdletBinding()] Param (
     [Parameter(Mandatory = $True, ValueFromPipelineByPropertyName = $True)] [string]$Path
    )

    [byte[]]$byte = get-content -Encoding byte -ReadCount 4 -TotalCount 4 -Path $Path

    if ( $byte[0] -eq 0xef -and $byte[1] -eq 0xbb -and $byte[2] -eq 0xbf )
    { Write-Output 'UTF8' }
    elseif ($byte[0] -eq 0xfe -and $byte[1] -eq 0xff)
    { Write-Output 'Unicode' }
    elseif ($byte[0] -eq 0 -and $byte[1] -eq 0 -and $byte[2] -eq 0xfe -and $byte[3] -eq 0xff)
    { Write-Output 'UTF32' }
    elseif ($byte[0] -eq 0x2b -and $byte[1] -eq 0x2f -and $byte[2] -eq 0x76)
    { Write-Output 'UTF7'}
    else
    { Write-Output 'ASCII' }
}

And using this to re-encode as ASCII:

If ((Get-FileEncoding -Path $file) -ine "ascii") {
    [System.Io.File]::ReadAllText($file) | Out-File -FilePath $file -Encoding Ascii
}

How to change the ps1 file in Unicode BigEndian to ASCII?

1 Answers1