non-ASCII characters in Powershell scripts

Question

I need to do some file operations in a file structure that contains special non-latin characters. Powershell crashes, when I try to use any of those characters.

For example, this doesn't work:

$TestPath = "C:\Examples\Folder_1ĀČ\"
$ExampleFileName = "Test.txt"

Copy-Item ($PSScriptRoot + "\" + $ExampleFileName)  -Destination ($TestPath) -Force

But this works:

$TestPath = "C:\Examples\Folder_1AC\"
$ExampleFileName = "Test.txt"

Copy-Item ($PSScriptRoot + "\" + $ExampleFileName)  -Destination ($TestPath) -Force

I tried debugging with

Write-Output $TestPath

And result returned in the console was:

C:\Examples\Folder_1Ä€Ä\

Is it possible to use powershell with paths containing these characters? How can I do that?

Answer 1

It looks as you have a hassle with your codepage in Powershell. Check and switch to UTF-8.

Have a look at this:
StackOverflow: Changing PowerShell's default output encoding to UTF-8

---

Update

As you write

I found out that -encoding default works for my case

Default is your system codepage.
The simplest way to display it, just execute:

chcp

=> What is your codepage?

---

I suppose you've a small typo in you question:

You write:        C:\Examples\Folder_1Ä€Ä\
I would expect:   C:\Examples\Folder_1ĀČ\

These are typical character code translation problems, when interpreting UTF8 character's binary encoding by a local ANSI codepage. Ā => Ä€ and Č => ÄŒ

Please note, it is of interest

• Which codepage is using your environment?
• Which character encoding (codepage) is using the cmdlet's text?
• Which encoding is used by Powershell for reading the cmdlet's text and for executing it.

Seeing your info updates the logic says:

1. As you can successfully execute the script with -encoding Default
   => Your script has been stored using your local codepage.

2. As not using -encoding Default results in "extended" characters:
   => Powershell assumes UTF8 as encoding, and

   • converts the read binary values of the file to correct UTF8 (changing the characters ĀČ proper UTF8 coded characters)
   • but finally the converted characters binary representation is interpreted using the local ANSI codepage.
     The result is ĀČ, as the characters ĀČ are 2-byte-encoded in UTF8.

---

As a consequence you should take care that all your environments (also your GUI editor) and Powershell's default are set to the same codepage.

Regarding this

PowerShell is now cross-platform, via its PowerShell Core edition, whose encoding - sensibly - defaults to BOM-less UTF-8, in line with Unix-like platforms.

(citation from link above)

I would suggest to migate everything to UTF8, so -encoding Default becomes the same as -encoding UTF8.
But be sure to do brief testing of your stored file-/directory-names and content, as currently they all are written using your local ANSI codepage.

In the meantime you have to tell Powershell, by -encoding Default not to assume your cmdlet is stored using UTF8.

---

How do I use this encoding for other functions like Copy-Item?

By using

mycmdlet.ps1 -encoding Default

You tell Powershell to read everything with your currently used local ANSI codepage. So everything that is handled by the commands will fit to that.
Wenn something comes in or leaves the cmdlet processing (because it's read or written) the system's codepage (local ANSI) will be used and also there should be everything OK.